Achievement 
Testing Program 
Provincial Report 


June 1993 Administration 


Grade Level 3 Language Learning 
Grade 6 Social Studies 
Grade 9 Science 


Alberta 


EDUCATION 


Achievement 
Testing Program 
Provincial Report 


June 1993 Administration 


Alberta Education 
Student Evaluation Branch 


ATA LIBRARY 
110}0 - 142 Street NW 
Edmonton, AB 


TSN 2R1 AE-GCUK 


Alberta Education Cataloguing in Publication Data 

Alberta. Alberta Education. Student Evaluation Branch. 
Achievement testing program, provincial report: June 1993 
administration. 


ISBN 0-7732-1171-3 


1. Achievement tests — Alberta. 
2. Educational tests and measurements — Alberta. 
I. Title II. Title: Provincial report: June 1993 administration. 


LB3060.3.A333 1994 | 371.26 


This document was written primarily for: 


General Public 


Distribution: Superintendents of Schools * School Principals and Teachers * The Alberta Teachers’ 
Association * Alberta School Boards Association ¢ Alberta Education ¢ General Public upon Request 


Copyright 1994, the Crown in Right of Alberta, as represented by the Minister of Education, Alberta 
Education, Student Evaluation Branch, 11160 Jasper Avenue, Edmonton, Alberta, T5K OL2. All rights 
reserved. Additional copies may be obtained from the Student Evaluation Branch, 427-0010. 


xi 


101 
107 
111 


115 


117 


121 


Contents. 


Message from the Director 

Section 1: Summary of Achievernent Test Results 
Section 2: Grade Level 3 Language Learning 
Section 3: Grade 6 Social Studies 

Section 4: Grade 9 Science 

Section 5: Achievement by Gender 

Section 6: Achievement by Age 

Section 7: Achievement by Grade Level 

Section 8: Achievement Over Time 

Section 9: Age 9 Achievement 

Appendix A: Standards 

Appendix B: Public Review of Standards and Results 


Appendix C: Guidelines for Interpreting Results 


Appendix D: Reporting to Parents—Answers to Frequently 


Asked Questions 
Appendix E: Developing Achievement Tests 


Questionnaire 


ttt 


31 


31 


54 


+ List of Figures 


Section 1 


Figure 1-1 
Student Participation in Achievement Tests 


Figure 1-2 
Percentage of Students Achieving Acceptable Standard 
on the Total Test 


Figure 1-3 
Percentage of Students Achieving Standard of Excellence 
on the Total Test 


Section 2 


Figure 2-1 

Grade Level 3 Language Learning 

Percentage of Students Achieving Acceptable Standard 
on the Total Test and on Components of the Test 


Figure 2-2 

Grade Level 3 Language Learning 

Percentage of Students Achieving Standard of Excellence 
on the Total Test and on Components of the Test 


Section 3 


Figure 3-1 

Grade 6 Social Studies 

Percentage of Students Achieving Acceptable Standard 
on the Total Test and on Components of the Test 


Figure 3-2 

Grade 6 Social Studies 

Percentage of Students Achieving Standard of Excellence 
on the Total Test and on Components of the Test 


Section 4 

Figure 4-1 

Grade 9 Science 

Percentage of Students Achieving Acceptable Standard 
on the Total Test and on Components of the Test 


55 


66 


66 


77 


78 


78 


80 


81 


103 


Figure 4-2 

Grade 9 Science 

Percentage of Students Achieving Standard of Excellence 
on the Total Test and on Components of the Test 


Figure 4-3 

Grade 9 Science 
Percentage of Students Achieving At or Beyond Grade 9 
Expectation (PBA), by Activity 


Figure 4-4 

Grade 9 Science 

Percentage of Students Achieving At or Beyond Grade 9 
Expectation (PBA), by Number of Activities 


Section 5 


Figure 5-1 
Number of Achievements Tests Written by Gender 


Figure 5-2 
Number and Percentage of Students Achieving Acceptable 
Standard on the Total Test by Gender 


Figure 5-3 
Number and Percentage of Students Achieving Standard 
of Excellence on the Total Test by Gender 


Section 6 


Figure 6-1 
Percentage of Students Achieving Acceptable Standard 
on the Total Test by Age 


Figure 6-2 
Percentage of Students Achieving Standard of Excellence 
on the Total Test by Age 


Section 7 


Figure 7-1 
Distribution of Grade Level Achievement 


Appendix A 


Figure A-1 
Process Model for Standard Setting: Achievement Testing 
Program 


vi 


10 


11 


12 


13 


13 


18 


20 


List of Tables- 


Section 2 


Table 2-1 
Grade Level 3 Language Learning: Achievement Test 
Blueprint—Part A: Writing 


Table 2-2 
Grade Level 3 Language Learning: Achievement Test 
Blueprint—Part B: Reading 


Table 2-3 
Grade Level 3 Language Learning: Student Participation 


Table 2-4 

Grade Level 3 Language Learning: Students Achieving 
Standards on the Total Test and on Components 

of the Test 


Table 2-5 ; 
Grade Level 3 Language Learning: Percentage 
Distribution of Schools Meeting Provincial 
Achievement Standards on the Total Test 


Table 2-6 

Grade Level 3 Language Learning: Percentage 
Distribution of Scores by Reporting Category—Part A: 
Writing 


Table 2-7 
Grade Level 3 Language Learning: Raw Score Results by 
Reporting Category—Part B: Reading 


Table 2-8 
Grade Level 3 Language Learning: Results for Individual 
Multiple-Choice Questions 


Table 2-9 

Grade Level 3 Language Learning Performance-Based 
Assessment: Percentage Distribution of Scores by 
Reporting Category—Reading Component 


Table 2-10 

Grade Level 3 Language Learning Performance-Based 
Assessment: Percentage Distribution of Scores by 
Reporting Category—Writing Component 


21 


24 


28 


29 


30 


32 


33 


34 


35 


36 


37 


Table 2-11 

Grade Level 3 Language Learning: Survey on Attitudes 
Toward Language Learning—Percentage Distribution of 
Student Responses 


Table 2-12 

Grade Level 3 Language Learning: Teacher Survey on 
Instructional Practices and Learning Environments— 
Percentage Distribution of Teacher Responses 


Section 3 


Table 3-1 
Grade 6 Social Studies: Achievement Test 
Blueprint—Part A: Multiple Choice 


Table 3-2 
Grade 6 Social Studies: Achievement Test 
Blueprint—Part B: Written Response 


Table 3-3 
Grade 6 Social Studies: Student Participation 


Table 3-4 

Grade 6 Social Studies: Students Achieving 
Standards on the Total Test and on Components 
of the Test 


Table 3-5 

Grade 6 Social Studies: Percentage Distribution of 
Schools Meeting Provincial Achievement 
Standards on the Total Test 


Table 3-6A 

Grade 6 Social Studies: Results for Individual 
Multiple-Choice Questions 

Topic A—Local Government 


Table 3-6B 

Grade 6 Social Studies: Results for Individual 
Multiple-Choice Questions 

Topic B—Greece: An Ancient Civilization 


Table 3-6C 

Grade 6 Social Studies: Results for Individual 
Multiple-Choice Questions 

Topic C—China: A Pacific Rim Nation 


Table 3-7 
Grade 6 Social Studies: Raw Score Results by Multiple- 
Choice Reporting Category 


viii 


38 


42 


46 


49 


52 


53 


55 


56 


56 


57 


60 


Table 3-8 
Grade 6 Social Studies: Percentage Distribution of Scores 
by Written-Response Reporting Category 


Table 3-9 

Grade 6 Social Studies: Student Survey on Performance- 
Based Assessment—Percentage Distribution of 

Student Responses 


Table 3-10 

Grade 6 Social Studies: Teacher Survey on Performance- 
Based Assessment—Percentage Distribution of 

Teacher Responses 


Table 3-11 
Grade 6 Social Studies: Teacher Questionnaire—Contexts 
for Learning, Percentage Distribution of Responses 


Table 3-12 
Grade 6 Social Studies: Student Questionnaire—Contexts 
for Learning, Percentage Distribution of Responses 


Section 4 


Table 4-1 
Grade 9 Science: Achievement Test Blueprint 


Table 4-2 
Grade 9 Science: Student Participation 


Table 4-3 
Grade 9 Science: Students Achieving Standards on 
the Total Test and on Components of the Test 


Table 4-4 

Grade 9 Science: Percentage Distribution of 
Schools Meeting Achievement Standards on the 
Total Test 


Table 4-5 
Grade 9 Science: Raw Score Results by 


Reporting Category 


Table 4-6 
Grade 9 Science: Results for Individual 
Multiple-Choice Questions 


Table 4-7 
Grade 9 Science: Results for Individual 
Numerical-Response Questions 


65 


66 


68 


69 


71 


79 


86 


‘87 


8&8 


98 


98 


Table 4-8 
Grade 9 Science: Performance-Based Assessment 
Activities 


Table 4-9 
Grade 9 Science: Performance-Based Assessment 
Results 


Table 4-10 
Grade 9 Science: Relationship Between Achievement Test 
Results and Performance-Based Assessment 


Table 4-11 pa 
Grade 9 Science: Contexts for Learning—Percentage 
Distribution of Student Responses 


Table 4-12 
Grade 9 Science: Contexts for Learning—Percentage 
Distribution of Teacher Responses 


' Section 6 


Table 6-1 
Number of Achievement Tests Written by Age 


Section 8 


Table 8-1 
Design of the Achievement-Over-Time Tests: 
Number of Questions from Previous and Current Tests 


Table 8-2 
Comparison of AOT Sub-group Results with 1993 


. Provincial Results 


Table 8-3 
Achievement-Over-Time Comparison of Descriptive 
Statistics 


Section 9 


Table 9-1 
Age 9 Study, Language Learning: Percentage of Students 


Achieving Level of Performance on Both Components 
of the Test 


Table 9-2 

Age 9 Study, Language Learning: Percentage of Students 
Achieving Level of Performance by Grade— 

Reading Component 


98 


104 


105 


105 


Table 9-3 

Age 9 Study, Language Learning: Percentage of Students 
Achieving Level of Performance by Grade— 

Writing Component 


Appendix A 


Table A-1 
Grade Level 3 Language Learning: Assessment and 
Achievement Standards 


Table A-2 © 
Grade 6 Social Studies: Assessment and 
Achievement Standards 


Table A-3 
Grade 9 Science: Assessment and Achievement 
Standards 


| 
Message from 


the Director 


V 


I am pleased to present the annual 
report on provincial achievement 
testing for the June 1993 
administration. Students were 
assessed in Grade 3 language 
learning, Grade 6 social studies, 
and Grade 9 science. 


As in past years, the information in 
this report is based only on the 
students in the province who were 
assessed. However, approximately 
10 percent of the students in each 
grade did not write the test 

(page 1), so we have no information 
about these students’ achievement 
in relation to provincial standards. 
Beginning in the 1994-95 school 
year, all students registered in 
grades 3, 6, and 9 programs are 
expected to write the tests or be 
reported as not being able to 
achieve standards. Following the 
1995 adminstration, we will be able 
to provide more comprehensive 
information to schools and to report 
to the public on the achievement of 
all students, but our new inclusion 
policy is likely to affect the 
percentage of students who meet 
provincial standards. 


School level results suggest 
declining performance across 
grades 3, 6, and 9 as evident in the 
percentage of schools meeting the 
provincial achievement standard. 
As shown on page 56, Grade 9 level 
data show that a satisfactory 
percentage of students met the 
achievement standard in only two 
out of three schools. This indicates 
that not enough Grade 9 students 
know their science in one-third of 
our schools. The information in this 


report, together with detailed school 
data provided separately, should 
help school administrators, 
teachers, and the community to 
identify what needs to be done to 
improve learning in their schools. 


Again this year, the assessments 
included performance tasks at each 
grade level. These tasks extend the 
range of areas assessed and 
generally raise additional questions 
of importance for classroom 
instruction. For example, the Grade 
3 results, presented in Section 2, 
show that fewer students clearly 
met the provincial standard for 
writing in the performance 
assessment than on the written 
component of the achievement test. 
This type of extended writing task 
may be more useful in the context of 
classroom assessment, as a means 
of gaining insights about a 
student’s learning processes in 
writing for diagnostic purposes, 
since it does not add much to the 
measure of student’s writing 
competence at the end of Grade 3. 


As part of the 1993 testing, a 
special study was done with 
students aged 9 to investigate the 
merit of age-based testing. The 
study included a survey of teachers 
on the desirability of assessment by 
age. Although the results show 
some useful information about age 
and grade level performance, 
assessment by age proved 
disruptive for some schools. The 
study supports our decision to 
continue with our grade-based 
approach to assessment. See 
Section 9 for more detail. 


The provincial assessments and 
special studies can only be carried 
out with the help of teachers, 
principals, and superintendents. 
On behalf of the staff at the 
Student Evaluation Branch, I want 
to express our appreciation for this 
help. I trust that the enclosed report 
will be interesting to all of you, and 
I hope it will assist you in reflecting 
on your instructional programs and 
encouraging all students to meet 
high standards of achievement. A 
questionnaire is included at the end . 
of the report, and we welcome your 
feedback. 


lth 


Frank G. Horvath, Director 
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Section 1 


Summary of 
_ Achievement Test Results 


Parents, educators and the general 
public need to know how well 
Alberta students are achieving in 
relation to provincial standards. 


The 1993 achievement test results* 
show what students in grades 3, 6, 
and 9 know and can do in Grade 
Level 3 Language Learning, Grade 
6 Social Studies, and Grade 9 
Science. 


Knowledge of what is possible 
produces new enthusiasm, 
raises sights, establishes new 
challenges and ultimately can 
improve personal and societal 
performance.** 


This first section of the report 
describes certain broad 
characteristics of the student 
population who wrote the 
achievement tests and provides 
answers to the following questions: 


¢ How many students wrote 
the achievement test for 
their grade and how does 
this compare to 1992? 


¢ What percentage of the 
students achieved the 
acceptable standard 
according to criteria set 
by Alberta Education? 


What percentage of the 
students achieved the 
standard of excellence 
according to criteria set 
by Alberta Education? 


Student Participation 


How many students wrote the 
achievement test for their 
grade and how does this 
compare to 1992? 


In June 1993, principals reported 
a total population of 116 296 
students in the grades tested. 
There were 42 837 students in 
Grade 3, 39 097 students in 

Grade 6, and 34 362 students in 
Grade 9. Of the total population, 
102 702 wrote achievement tests. 
This means that 88.3% of students 
reported by principals to be 


enrolled in schools in June wrote 
the tests. The remaining 13 594 
students were exempted from 
writing or were absent the day the 
tests were administered. 


Figure 1-1 shows student 
participation data for June 1993 
and for June 1992. This report does 
not take these absent or exempted 
students into account in answering 
the questions about the percentage 
of students who achieved the 
acceptable standard and the 
standard of excellence. 


*In some cases, the percentages in a chart may not add to 100 because of rounding. 


**Learning Mathematics/Learning Science, International Assessment of Educational Progress, Educational Testing Service, 


February 1992. 


Figure 1-1 


Student Participation in Achievement Tests* 


June 1992 and June 1993 
Grade 3 


Grade 6 


100 


0 
1992 1993 
N= 41 696 N= 42 837 
Social English 
Studies Language 
Leaming 


a 
Students writing, June 1992 


[il Students absent or exempt, June 1992 


Wd 


1993 


1992 
N=37 837 N=39097 
English Social 
Language Studies 
Arts 


1993 


1992 
N= 34 386 N= 34 362 
Mathematics Science 


i Students writing, June 1993 


4, 
Students absent or exempt, June 1993 


*The number of students writing the tests includes students in the regular English program, and Francophone and French Immersion students 
who wrote the achievement test either in English or in French translation. 
N= number of students reported by principals to be enrolled in June. 


Observations and 
Discussion 


In 1993, the percentage of students 
who wrote the test was close to 
90% for each of the three grades. 
The percentage for Grade 6 was 
the highest (89.7%) compared to 
87.5% for Grade 3 and 87.7% for 
Grade 9. 


In terms of absolute numbers, 
Grade 3 had the highest number of 
students (37 500) writing the test, 
followed by students in Grade 6 
(35 062) and students in Grade 9 
(30 140). This trend corresponds 
with the trend in student 
enrollment by grade reported as of 
September 30. 


The total number of grades 3, 6, 
and 9 students who wrote the tests 
increased from 97 315 in 1992 to 
102 702 in 1993. This represents an 
increase of 5.5%. The increase 


occurred at each grade level in both 
absolute numbers and percentages. 
The percentage of students who 
were absent or exempted from 
writing the tests decreased in 1993 
compared to the percentage for 
1992. The 1993 overall 
participation rate of 88.3% is an 
improvement over the 1992 
participation rate of 85.4%. 


Results in Relation 

to Standards for 
Students Who Wrote the 
Achievement Tests 

What percentage of the 
students achieved the 
acceptable standard according 


to criteria set by Alberta 
Education? 


What percentage of the 
students achieved the standard 
of excellence according to 


‘criteria set by Alberta 
Education? 


Results are reported in terms 

of three related but different 
standards: curriculum standards, 
assessment standards, and 
achievement standards. 


Curriculum standards are the 
broad learnings, sequenced into 
grades, that students are expected 
to have achieved. They include 
statements of knowledge, skills, 
and attitudes against which 
student achievement is to be 
judged. 


Assessment standards are the 
scores to be achieved by a student 
on a specific test or part of a test 
before the performance of that 
student is judged to be 
“acceptable” or “excellent” relative 
to the curriculum standards. 


Alberta Education reports the 
results for the achievement tests in 
relation to an acceptable standard 
and a standard of excellence. 


Achievement standards state 
what percentage of students at a 
given grade in school is expected to 
achieve the acceptable standard or 
the standard of excellence. These 
achievement standards apply to 
school, jurisdiction, and provincial 
performance. 


Assessment and achievement 
standards for the 1993 achievement 
tests were based on 
recommendations from a 
curriculum and test development 
committee, a test review 
committee, a public advisory 
committee, and experienced 
teachers who applied standard- 
setting procedures to the tasks 
under the guidance of the Analytic 
Services Unit of the Student 
Evaluation Branch. An outline of 
the processes followed is provided 
in Appendix A. Appendix B 
describes the public review of 
standards and results. 


Guidelines for interpreting the 
1993 results are given in 
Appendix C, and Appendix D 
provides answers to a number of 
questions frequently posed by 
parents. Appendix E describes the 
development of achievement tests. 


In this section, results are reported 
for the total test. Results for the 
major components of the tests can 
be found in sections 2, 3, and 4. 


From discussions with educators, 
test development specialists and 
curriculum specialists, and based 
on our experience with measuring 
student achievement according to 
the expectations in the Program of 
Studies, we expect 85% of students 
who write the test to achieve the 
acceptable standard and 15% of 


Figure 1-2 
Percentage of Students Achieving Acceptable Standard* on the 
Total Test 
June 1993 
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Language Learning 
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N= 37 500 


Social Studies Science 
(Gr. 6) (Gr. 9) 
N= 32 630 N= 28613 


*tncludes students achieving the standard of excellence 


**85% of students who wrote the test were expected to achieve the acceptable standard on 
the total test. 


Figure 1-3 


ES 


Percentage of Students Achieving Standard of Excellence on the 


Total Test 
June 1993 


20 


Language Learning 
(Gr. Level 3) 
N= 37 500 


Social Studies Science 
(Gr. 6) (Gr. 9) 
N= 32 630 N= 28613 


*15% of students who wrote the test were expected to achieve the standard of excellence on 


the total test. 


students who write the test to 
achieve the standard of excellence. 


Figures 1-2 and 1-3 present 

the percentage of students who 
achieved the acceptable standard 
and the standard of excellence 
based on their total test scores. 


Provincial results for Grade 3 in 
1993 are based on test scores 
achieved by all students who wrote 


the test, including Francophone 
and French Immersion students. 


Provincial results for grades 6 and 
9 in 1993 are based on test scores 
achieved by students who were in 
the regular English program. 
Results for Francophone and 
French Immersion students were 
reported directly to participating 
schools and jurisdictions and are 
not calculated in provincial results. 


3 


General Observations 
on Results 


Of the three grades and subjects 
tested in 1993, the results in 

Grade Level 3 Language Learning 
were closest both to reaching 
provincial achievement standards 
and to being judged satisfactory. In 
Grade Level 3 Language Learning, 
results based on the total test 
scores revealed that the percentage 
of students achieving the 
acceptable standard was slightly 
lower than expected, and the 
percentage achieving the standard 
of excellence was marginally lower 
than expected. 


Results for Grade 6 Social Studies 
were disappointing. The 
percentages of students achieving 
standards were lower than 
expected for both levels. 


Grade 9 Science results were 
mixed. The percentage of students 
achieving the acceptable standard 
was below expectation. The 
percentage of students who 
achieved the standard of excellence 
was higher than expected. 


pi 


os, + | ; 
Section.2 


Ls 
Grade Level 3 
Language. Learning 


In 1993 a number of assessments 
were carried out in Grade Level 3 
Language Learning. The 
achievement test was administered 
to students province-wide. A 
Performance-Based Assessment 
was administered to a sample of 
Grade Level 3 Language Learning 
students. As well, a sample of 
students and teachers participated 
in a study to examine the 
relationship among various 
contexts for learning and 
achievement. The results from all 
these assessments follow. 


Achievement Test 
General Description 


The Grade Level 3 Language 
Learning Achievement Test had 
two parts. Part A: Writing was a 
65-minute assignment consisting of 
a picture and a prompt for writing. 
The assessment was designed to 
encourage students to use the 
writing process. Part B: Reading 
was a 50-minute reading test 
consisting of 40 multiple-choice 
questions based on nine reading 
selections. 


Statistics for the total test and for 
the components are based on the 
results achieved by 37 500 
students: 34 607 wrote the regular 
form and 2 893 participated in the 
achievement-over-time study, as 
shown in Table 2-3. This section of 
the report answers the following 
questions: 


*® How many Grade Level 3 
students wrote each form of the 
test or were absent or exempt? 


© What percentage of students who 
wrote the Grade Level 3 
Language Learning achievement 
test achieved the acceptable 
standard? 


© What percentage of students who 
wrote the Grade Level 3 
Language Learning achievement 
test achieved the standard of 
excellence? 


© What did Grade Level 3 students 
know and what could they do in 
Language Learning? 


© What parts of the Language 
Learning curriculum were 
difficult for Grade Level 3 
students? 


Summary of Results 


Results show that 82.2% of 
students who wrote the test 
achieved the acceptable standard 
and 14.1% achieved the standard 
of excellence on the total test. 
These results were slightly lower 
than expected. 


Content of the Test 


Part A: Writing provided the 
students with a picture and a brief 
explanation as a prompt for their 
writing. Students chose the format 
(narrative, letter, or diary/journal 
entries) that would allow them to 


do their best writing while using 
their imagination and background 
experience. This part of the test 
was scored in three categories: 
Content and Development, Use of 
Language, and Conventions. 
Content and Development was 
weighted to be worth twice as 
much as each of the other 
categories. 


Part B: Reading questions were 
based on reading selections from 
fiction, non-fiction, poetry, and 
visual media. These selections 
were chosen to reflect the varying 
interests and ability levels of 
students in Grade Level 3 
classrooms. Canadian material was 
used wherever possible. 


Test Blueprint 


The test blueprint for Part A: 
Writing shows the reporting 
categories (scoring guide), a 
description of the writing 
assignment, and the emphasis of 
communication that was assessed. 


Table 2-1 
Grade Level 3 Language Learning 
Achievement Test Blueprint 


Part A: Writing 
June 1993 
Reporting Category (Scoring Guide) Description of Writing Assignment Standards 
Content and Development* —meets or exceeds 
Events and/or actions should be plausible and appropriate The writing assignment follows a writing the standard of 
to the student's purpose for communicating. The student prompt that is read aloud to the students. excellence 
should be able to describe characters and settings that are The assignment allows the student to select —approaches the 
appropriate within the context or terms of reference the format that would best fit his/her standard of 
established by the student. approach to the prompt. excellence 
—clearly meets the 
Use of Language acceptable 
The student should be able to use words and expressions standard 
effectively in writing. — approaches the 
acceptable 
Conventions (Using the conventions of language correctly standard 
and effectively) —clearly below the 
The student should be able to communicate clearly in writing acceptable 
by adhering to appropriate spelling, grammar, punctuation, standard 
and capitalization. INS — Insufficient 


* This category is weighted to be worth twice as much as each of the other two. 


The test blueprint for Part B: 
Reading shows the reporting 
categories and genre under which 
questions are classified. 


Table 2-2 


Grade Level 3 Language Learning 
Achievement Test Blueprint 


Part B: Reading 
June 1993 
Question Distribution by Genre 
Organizational Specific Fiction Approx. | Approx. 
Framework Learner Visual Non- Number | Percent 
(Functions) General Concept Statement Expectations; Stories | Poetry Media Fiction {| ofitems | of Test 
—~— }--—— 
Exploring Concept D: Knowing how ideas D.1, D.2, D.3, 28 32 14,16 | 8, 37, 38, 9 22.5 
and information can be D.4, D.5, D.6, 39, 40 
organized and presented D.7 
contributes to the enhanced 
understanding and 
communication of ideas. 
Constructing |Concept G: The ability to make G.1, G.2, G.3, | 1,19, 20, | 33, 34 7 17.5 
associations and connections is G.5, G.6 27, 31 
essential to the understanding 
and communication of meaning. 
Concept H: The ability to think H.1, H.3, H.6, | 2, 4,5, 18, | 35, 36 13,15 | 12, 22, 24 12 30 
analytically is necessary for H.9, H.10, 30 
critical reading, listening, and H.11, H.12 
writing.. 
‘| Concept |: The ability to 1.1, 1.3, 1.5 3, 21, 29 17 6, 7, 9, 10, 12 30 
synthesize is necessary to the : 11, 23, 25, 
understanding and retention of 26 
ideas and information. 
Total Items 14 5 5 16 40 100 
Total Percent 35 12.5 12.5 40 100 100 


Student Participation 

In June 1993, principals reported a 
total population of 42 837 students 
in Grade Level 3. Table 2-3 
presents the number and 


percentage distribution of students 
who wrote the Grade Level 3 
Language Learning Achievement 
Test, and who were absent or were 
exempted. In total, 87.5% of the 


students reported to be in Grade 
Level 3 in June 1993 wrote the 
Grade Level 3 Language Learning 
Achievement Test. 


Table 2-3 

Grade Level 3 Language Learning 
Student Participation 

June 1993 


Category 


_ “Total Number of § udents ini Regular Programs’ 


rote me Regular Form** 


Studen Who Wrote the Achiovernent-Ovet:Time Forms: hee 
=. Form'M: (Lavender) — Same a as the ‘Regul Form ear 

‘Form L (Red): ‘a 

orm K: (Bue). 


Students Absent 
Students Exempted from Writing 


Categories of Exemption: 


Special Needs Students 

Course Was Not Taught This Term 

English as a Second Language Students 
Other (as approved by the Superintendent) 


Pons 


Total Principals’ Reported Population 
Test Day, June 1993: 


Grade 3 Enrollment: September 30, 1992 


Number Percentage 
of Students of Students 
2 87800", 
994607, 
2003 
1018 os 
940i oe 
2 299 5.4 
3 038 7.1 
1 608 3.8 
61 0.1 
604 1.4 
765 1.8 
42 837 100.0 
42 637 
eee | 


*Provincial results are based on test scores achieved by all students who wrote the Grade Level 3 Language Learning test, including 


Francophone students, French immersion students, and students in other programs. 


“Regular Form refers to the unmodified 1993 test. The modified tests are described in Section 8 of this report. 


Results in Relation 
to Standards 


Achievement test results are 
reported in relation to assessment 
and achievement standards. The 
Provincial Assessment 
Standard is the lowest score on a 
test that a student must achieve 
for his/her performance to be 
judged “acceptable” and/or 
“excellent” in relation to curricular 
expectations. The Provincial 
Achievement Standard refers to 
the percentage of students writing 
the test who are expected to 
achieve the Provincial Assessment 
Standard. 


Figures 2-1 and 2-2 and Table 2-4 
show the percentage of students 
achieving the acceptable standard 
and the standard of excellence on 
the total test and on components of 
the test. The levels of performance 
in Grade Level 3 writing and 
reading were marginally lower 
than expected; however, 
significantly more students than 
expected achieved the standard of 
excellence for reading. 


When interpreting these findings, 
it is important to remember that 
they are based on the test scores of 
only a portion (87.5%) of the 
students reported to have been 
enrolled in Grade Level 3 in June 
1998, i.e., those students in regular 
programs who wrote the 
achievement test. These figures 
tell us nothing about the levels of 
achievement of students who were 
absent on the day of writing, or of 
those students who were exempted 
from writing. 


Figure 2-1 

Grade Level 3 Language Learning 

Percentage of Students Achieving Acceptable Standard on the 
Total Test and on Components of the Test 

June 1993 


Writing Skills Reading Skills Total Test 


*Provincial Achievement Standard—85% of the students in the province who wrote the test 
are expected to achieve the acceptable standard 


Figure 2-2 

Grade Level 3 Language Learning 

Percentage of Students Achieving Standard of Excellence on the 
Total Test and on Components of the Test 

June 1993 


30 
25. 


% 


Total Test 


“Provincial Achievement Standard—15% of the students in the province who wrote the test 
are expected to achieve the standard of excellence 


Table 2-4 
Grade Level 3 Language Learning 
Students Achieving Standards on the Total Test and on Components of the Test 


June 1993 


Reporting Category 
Standard of Excellence 

Total Test 

Writing Skills Only 

Reading Skills Only 


Provincial |: Provincial. 
Maximum | Assessment |Achie 
Possible Standard* 


Score (Raw Score) 


Acceptable Standard 
Total Test 

Writing Skills Only 
Reading Skills Only 


Students Achieving Assessment 
Standard 

Actual 

Number 


Expected 
Number 


Below Acceptable Standard 


100 79 
20 15 
40 33 a 
ee REIT 
100 48 
20 9 
40 19 
N/A N/A 
[ on Both Writing and Reading Skills 


“The Provincial Assessment Standard is a score determined by standard-setting procedures and is the lowest score a student must 
achieve for his/her performance to be judged “acceptable” and/or “excellent” in relation to curricular expectations. See Appendix A. 


“The Provincial Achievement Standard refers to the percentage of students expected to achieve the Provincial Assessment Standard. 


It should be noted that the actual 
percentages of students meeting 
standards on the total test (14.1 
and 82.2) are based on the 

37 500 students in the regular 
programs who wrote the test. 


If, however, the percentages are 
based on the total population 
reported by principals (42 837), the 
percentages achieving standards 
on the total test would be: 
12.3% meeting the standard of 
excellence, 
72.0% meeting the acceptable 
standard. 


If the percentages are based on the 
September 30, 1992, Grade Level 3 
enrollment (42 637), the 
percentages achieve the standards 
on the total test would be: 
12.4% achieving the standard of 
excellence, 
72.3% achieving the acceptable 
standard. 
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It is emphasized that the above 
percentages, based on total 
population and enrollment figures, 
present the lowest estimate of 
achievement. It is highly likely 
that some of the students who were 
absent, exempt, or not accounted 
for would have achieved standards. 
The absence of information on 
these students is nonetheless 
problematic. 


The number of students achieving 
the acceptable standard and the 
standard of excellence for each 
school was analyzed to determine 
whether schools were below, 
meeting, or exceeding provincial 
achievement standards. Schools 
classified as meeting provincial 
achievement standards were those 
for which the difference between 
the actual number of students and 
the expected number of students at 
or above standards was not 
statistically significant. Differences 


are only reported when there is a 
5% or smaller probability that a 
difference of that size could occur 
by chance. 


About one-quarter of schools were 
significantly below the provincial 
achievement standard. School | 
administrators and teachers 
should critically examine language 
learning achievement test results 
to determine what they are going 
to do to improve the level of 
student achievement. Although 
this is true for all schools, it is 
particularly important for those 
schools performing below the 
provincial achievement standard. 


Table 2-5 
Grade Level 3 Language Learning 
Percentage Distribution of Schools* Meeting Provincial Achievement Standards on the Total Test 


(N = 1 106) 
June 1993 


Standard 
Standard of Excellence 


Acceptable Standard 


Percentage Distribution of Schools 
Significantly Below Not Significantly Different Significantly Above 
Provincial Achievement From Provincial Provincial Achievement 
Standard Achievement Standard Standard 
TA 83.1 9.8 
23.1 69.0 7.9 


*Schools with fewer than five students are excluded because the statistical significance of the difference between the number achieving the 
standard and the number expected to achieve the standard when calculated and reported is not educationally meaningful. 


Results for 

Part A: Writing 

Results for Part A: Writing are 
most clearly understood in the 
context of the assignment students 
responded to and in the context of 
the scoring guides. The average 
raw score was 11.4 out of a possible 
20, with a standard deviation of 
3.1. Complete scoring guides are 


available from the Student 
Evaluation Branch, Alberta 
Education. 


Although the papers were scored 
on a one-marker system, 200 
randomly selected papers were re- 
marked so that a second set of 
scores was available to confirm 
scoring consistency. Of the scores 


awarded on the second reading, 
49.3% were identical to the original 
score on the same scale and 42.3% 
varied by only one point. The one- 
marker system produces results 
that are reasonably reliable for 
groups of 25 or more students. 
Achievement test scores, however, 
are less reliable for individual 
students. 
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The results presented in Table 2-6 
are best considered in terms of the 
percentage of students that 
markers judged to have presented 
work that clearly met the 
acceptable standard for any 
reporting category. It is possible to 
draw conclusions about local 
program strengths and weaknesses 
by comparing local results to 
provincial standards. Statistics 
presented in Table 2-6 are based 
on results achieved by 37 500 
students. 


It should be noted that writing 
standards in 1993 were higher 
than those of previous years. In 
1993, for example, to be awarded a 


3 a student’s performance was 
described as “clearly” meeting 
expectations, whereas in previous 
years a score of 3 reflected 
“satisfactory” performance. 


Nonetheless, the results in the 
three individual reporting 
categories bear close scrutiny. Of 
these, in “Content and 
Development” (students describe 
characters and settings that are 
appropriate for the context or 
terms of reference they have 
chosen), approximately 1 in 3 or 
32.3% were not able to meet the 
acceptable standard. In “Use of 
Language” (students use words 


and expressions effectively in their 


writing), 28.7% or slightly more 
than 1 in 4 did not meet the 
acceptable standard. The results for 
the “Conventions” category 
(students communicate clearly by 
adhering to appropriate spelling, 
grammar, punctuation, and 
capitalization) are similar to the 
“Content and Development” 
category. A large number of 
students, slightly more than 1 in 3 
or 35.8%, did not meet the 
acceptable standard. Students did 
better on the writing component of 
the achievement test than they did 
on the performance-based 
assessment (see Table 2-10). 


Table 2-6 


Grade Level 3 Language Learning 


Percentage Distribution of Scores by Reporting Category 


Part A: Writing 
June 1993 


Reporting Category 
Content and 
Standards Development Use of Language Conventions 
Meets or Exceeds the Standard of Excellence 29 44 
Approaches the Standard of Excellence 14.7 17.7 


SSCS eS 


Results for 

Part B: Reading 

Reporting Categories 

Table 2-7 shows the total marks 
possible and the provincial raw 
score results for the reporting 
categories of the reading portion of 
the Grade Level 3 Language 
Learning Achievement Test. 
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It is important to stress that the 
averages on the various reporting 
categories cannot be directly 
compared with one another. 
Rather, the results shown in Table 
2-7 can best be used in conjunction 
with parallel tables in the 
jurisdiction and school reports. 
Variations in patterns of students’ 
responses to questions can help to 


I OE ET OEE ET EE ET 


indicate strengths and weaknesses 
in local educational programs. 
Statistics presented in this table 
are based on results achieved by 
35 620 students (those who wrote 
the regular form and those who 
wrote Form M). 


Table 2-7 


Grade Level 3 Language Learning 
Raw Score Results by Reporting Category 


Part B: Reading 
dune 1993 
Number of Raw Score Raw Score 
Reporting Category Questions Average Standard Deviation 

Total 40 26.7 8.0 
Organization of Information 9 6.0 2.2 
Associating/Connecting 7 4.3 , 1.8 
Analyzing 12 8.4 2.5 
Synthesizing 12 8.0 2.7 
Stories 14 9.3 3.2 
Poetry 5 3.1 1.4 
Visual Media 5 3.8 1.2 
Non-Fiction 16 10.5 3.5 


Percentage of Students — question measures is specified. questions can help to indicate 
Choosing Each Alternative The questions are grouped by strengths and weaknesses in local 
Table 2-8 shows the percentage of reporting category. education programs. 

students who chose each 

alternative (A, B, C, and D) for The results shown in Table 2-8 can Statistics presented in Table 2-8 
each multiple-choice question. The best be used in conjunction with are based on results achieved by 
correct response for each question similar tables in the jurisdiction 35 620 students (those who wrote 
is identified with an asterisk and and school reports. Variations in the regular form and those who 
the curriculum standard each patterns of students’ responses to wrote Form M). 


Table 2-8 
Grade Level 3 Language Learning 
Results for Individual Multiple-Choice Questions** 


June 1993 
Distribution of Responses (%) 
Item Category Curriculum Standard 
A B c | D 
8 57.2* 41 6.8 30.8 (e) use headings to assist reading 
14 12.8 4.9 11.9 69.6* 10 use visual cues in conjunction with print to make 
sense of reading 
16 9.5 11.6 12.5 65.4* 10 locate author using experience with book elements 
28 12.0 11.5 64.4* 10.9 10 locate pertinent details about setting 
* correct answer. , Continued 


**The sum of the percentages for each question may be less than 100% because the No Response category is not included. 


Category Legend: |O—Identifying the Organization of Information AC—Associating/Connecting A—Analyzing Sl—Synthesizing Ideas 
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Table 2-8 (continued) 


Distribution of Responses (%) 


tem — 
A c D 
32 12.8 3.9 13.7 68.5" 
37 7.2 1.8 16.7 72.6" 
38 73.9* 74 8.6 8.3 
39 8.2 72.7* 11.8 5.6 
40 12.1 54.2" 12.2 19.7 
1 17.4 34.5 7.6 40.0* 
19 62.0* | 10.4 10.8 16.0 
20 | 203 13.2 14.2 51.6* 
27 8.2 | 77.9* 5.3 7.4 
31 5.0 6.3 77.5* 9.3 
73.5* 4.0 16.1 4.8 
26.9 12.2 51.3* 8.0 
2 13.7 79.0* 
4 75.1* 6.1 12.5 58 
5 | 63.8 7.9 10.1 17.6 
12 26.6 4.2 15.9 52.5* 
13 76.7" | 16.0 3.1 3.8 
15 3.2 5.1 87.6* 3.4 
18 10.5 3.6 82.1" 3.3 
22 17.0 9.5 4.0 68.6* 
24 9.2 15.0 58.7" 15.7 
30 55 10.4 5.2 77.2" 


Curriculum Standard 


attend to typographical features to assist reading 


attend to text features to assist reading 


use visual cues in conjunction with print to make 
sense of reading 


use headings to assist reading 


use visual cues in conjunction with print to make 
sense of reading 


identify the meaning of a word from its context 


decide the most likely meaning of a word using context clues 
relate pertinent details describing character motivation 


decide the most likely meaning of an expression 
using context clues 


identify connotations of words beyond their literal meaning 


identify connotation of a phrase beyond its literal meaning 
decide the most likely meaning of a word using context clues 
determine explicit relationships between characters 
recognize the explicit relationship between characters 


and events 


recognize the explicit relationship between characters 
and events 
determine the relationship between title and passage 


infer character motivation using pertinent details 


infer character’s feelings using pertinent details 


recognize the explicit relationship between characters 
and events 


determine the relationship among details in the passage 
recognize key details in the passage 


infer implicit relationships between characters and events 
using pertinent details 


*correct answer. 


Continued 


Category Legend: |O—Identifying the Organization of Information AC—Associating/Connecting A—Analyzing Si—Synthesizing Ideas 
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Table 2-8 (continued) 


Distribution of Responses (%) 


item Category Curriculum Standard 
A. B c D 
‘ ., . 
35 11.9 77.0* 4.2 §.3 A determine poet's attitude towards subject 
36 46.7 75 37.2" 6.5 A infer poet’s purpose for writing a poem 
3 2.8 8.0 86.1* 2.7 SI judge character motivation using pertinent details 
6 19.0 14.5 54.0* 12.0 Sl make a judgement about key details in the passage 
7 6.1 73.9* 9.6 9.8 Sl draw a conclusion by relating what she/he knows to new 
information in the passage 
9 5.3 9.1 70.8" 14.4 Sl make a generalization about characters using 
pertinent details 
10 81.1* 4.8 5.9 7.6 SI make a generalization about characters using 
pertinent details 
11 27.0 43 5.8 62.3" Sl judge character motivation using pertinent details 
17 81.8* 4.0 2.2 11.0 S| draw a conclusion by relating what she/he knows to 
new information in the passage 
21 15.6 14.0 54.1* 15.3 Sl draw a conclusion by relating what she/he knows to 
new information in the passage 
23 49.1* 7.4 36.5 5.9 Sl draw a conclusion by relating what she/he knows to 
new information in the passage 
25 9.6 78.0* 49 6.2 SI draw a conclusion by using information from the passage 
26 14.9 69.4" 45 9.5 SI make a generalization using given information 
29 26.5 40.4* | 10.6 20.7 $I draw a conclusion by relating what she/he knows to 


“correct answer. 


new information in the passage 


Category Legend: 1O—Identifying the Organization of Information AC—Associating/Connecting A—Analyzing Sl—Synthesizing Ideas 
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Examiner’s Observations 


Writing Component 

The overall quality of the writing 
in Language Learning in 1993 is 
better than that done in 1989. 
Teachers who marked the tests 
were very pleased with most of the 
papers. They observed specifically 
that students are strong in the 
following areas: planning, story 
structure, and content and 
development. The markers agreed 
that students, for the most part, 
showed genuine confidence in their 


17. What lesson does this story teach 


us? 
81.8 @ Be satisfied with the way you 
are. 
4.0 o Be sure to look before you 
leap. 
2.2 o The biggest dog was once a 
puppy. 
11.0 o Donkeys and dogs can be 
friends. 


1. What is the meaning of the 
underlined word seldom? 
17.4 o Many times 
34.5 o Often 
7.6 o Never 
40.0 @ Hardly ever 
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Reading Component 

The following is a discussion of 
specific areas of strength and 
weakness for students in Grade 3 
classrooms. Sample questions from 
the test are provided to highlight 
these areas for students achieving 
the acceptable standard and for 
those achieving the standard of 
excellence. For each sample 
question, the filled-in circle shows 
the keyed answer. The percentage 
of students choosing each 
alternative is also provided. 


writing and hence were willing to 
take the risks that resulted in a 
better product. 


Asmall number of students wrote 
superior papers. These were lively, 
imaginative, and technically very 
well written. The quality of these 
papers far exceeded the 
expectations for students in a 
Grade 3 classroom. 


Acceptable Standard 


For question 17, students had to determine the lesson or moral implied in 
the reading passage. Those students who achieved the acceptable standard 
were able to do this quite readily. 


In question 1, students achieving the acceptable standard but not the 
standard of excellence had difficulty inferring the meaning of a-word from 
its context in a sentence. 


The strengths of students who achieved the acceptable standard include 

an ability to 

* determine the lesson or moral implied in the reading selection (see 
question 17) 

* identify directly stated details of setting and character motivation 

¢ infer character motivation 

® use visual cues in conjunction with print to make sense of their reading 


However, many of these students did not do as well as expected in - 

* recognizing the meaning of words and phrases from the context of the 
material (see question 1) 

¢ making judgements about the author’s purpose for writing 

* clearly understanding selections that were not the usual story form: 
poems, recipes, and experiments 


26. People would MOST LIKELY 
make their own crystal mountains 
because 


14.9 0 itis fun to watch them in the 
dark 
69.4 @ crystals are pretty and 
interesting 
4.5 o itis good to eat them 
9.5 o crystals are quick and easy to 
grow 


7. The sticks are placed in the 
mixture when it is almost frozen 
so that the 


6.1 0 sticks will not break 
73.9 e sticks will stay in place 
9.6 o mixture freezes better 
9.8 o mixture will not separate 


Issues 

During the 1993 marking session, 
teachers were asked to comment 
on both parts of the test. Teachers 
were mostly in favour of the two 
changes that were made in the 
administration of the Language 
Learning Test: the assessment 
was written over two days, the 


writing component on the first day _ 


and the reading component on the 
second day, and students were 
allowed to discuss the writing 
prompt in groups of two to four. 


Standard of Excellence 


Students achieving the standard of excellence had no difficulty with the 
following sample questions that accompanied experiments, a written form 
which students achieving the acceptable standard (but not the standard of 
excellence) found more difficult. 


Question 26 required students to synthesize information given in the 
experiment to make a judgement about what people might do. 


Question 7 required students to relate what they know to new 
information presented in the passage and to draw conclusions. 


Clearly, students who achieved the standard of excellence (28.8%) had 
very few difficulties with this assessment. These students could 


© synthesize information from the selection to make judgements/ draw 
conclusions (see question 26) 

* recognize the meanings of words and phrases from the context of the 
material 

° make judgements about the author’s purpose for writing 

© make judgements about character motivation 

* understand clearly the selections that were not the usual story form: 
poems, recipes, and experiments 


Teachers thought that these who have a conventional system in 


changes were positive because they 
helped students do their best work. 


One area of concern emerged from 
the multiple-choice reading 
component. Questions from the 
catalogue card selection caused 
some misgivings for teachers in 
schools with an automated library 
system. Teachers thought that 
those students familiar with an 
automated system would be 
disadvantaged compared to those 


their school. In fact, analysis of the 
four questions pertaining to the 
catalogue card indicates that there 
was no advantage to students 
familiar with more conventional 
systems, and on the whole, 
students in the province did very 
well on those four questions (see 
questions 37 to 40). 
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Performance-Based 
Assessment 

As part of Alberta Education’s 
broadened assessment initiatives, a 
sample of 594 Grade 3 students 
from across the province 
participated in the Language 
Learning Performance-Based 
Assessment (PBA). This activity- 
based assessment using actual 
books was developed by Grade 3 
teachers to assess a broader range 
of reading and writing skills and to 
support classroom instruction. 
Students remained together as a 
class in their regular classroom 


setting, with both their teacher 
and the assessment administrator 
present. By engaging students in 
hands-on book reading followed by 
writing, we hoped to obtain 
valuable information about how 
well students explore, construct, 
and communicate meaning. 


Reading Component 
General Description 

The reading component required 
students to choose a book from a 
given selection. A range of titles 
was provided, all considered by 
Grade 3 teachers to be appropriate 


for their students. Several titles 
were included for students who 
needed reading material at a 
slightly lower level. After reading, 


’ students responded toa series of 


open-ended questions. Students 
were given as much time as was 
needed to complete the 

assessment. Copies of the student 
response sheets and scoring guides 
are available from Student 
Evaluation Branch, Alberta 
Education. Provincial results from 
the reading component a are shown 
in Table 2-9. 


Table 2-9 
Grade Level 3 Language Learning Performance-Based Assessment 
Percentage Distribution of Scores by Reporting Category 


Reading Component 
June 1993 
Reporting Category 
Retelling Main Personal 
Score * Predictions Story Structure Events Connections Opinions 
5 13 05 56 
108 
345 


*A score of 3 or above indicates that students have met or exceeded the acceptable standard. Scores within the shaded area (2 or 
below) indicate that students have not met the acceptable standard. 
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Examiner’s Observations 


The assessment was divided into six 
scoring categories (Predictions, 
Story Structure, Main Events, 
Characters, Pergonal Connections, 
and Opinions) and the way in which 
students met the scoring criteria 
varied depending on the category. 


In all, three out of four of students 
achieved the acceptable standard 
for Grade Level 3. Those students 
were able to express predictions 
related to the book cover, 
demonstrate an understanding of 
story events and sequence, and 
identify most of the story’s main 
events. They were also able to 
describe the cause of a story 
character’s behaviour, make a 
personal connection to what they 
read, and express and loosely 
support an opinion about the book. 
Students who are achieving beyond 
Grade Level 3 expectations were 
able to make predictions based on a 
story problem or theme, describe in 
detail all key features of a given 
story, and express main events of a 
story in a clear and concise manner. 
They were also able to describe 
internal and external character 
motivation, make personal 
connections related to the story 
problem or theme, and supply an 
opinion about the story along with 
justification. 


One quarter of the students did not 
achieve the acceptable standard for 
Grade Level 3. Those students had 
difficulty making predictions 
related to the book. They could 
retell only some story events and 
often included supporting details in 
place of main events. They were 
able to describe what story 
characters did without referring to 
the cause of the behaviours, identify 
but not explain a personal 
connection to the story, and supply 
an opinion about the book without a 
reason. 


In the category Opinions, a high 
percentage of students, 87%, were 
able to express and support 
opinions at an acceptable standard. 
The category with the next highest 
percentage of students achieving 
an acceptable standard was 
Predictions, where two-thirds of 
the students were able to make 
plausible predictions. Students had 
the most difficulty in the Personal 
Connections category, where they 
were required to connect what they 
read to their personal experiences. 
Only 48% of them achieved an 
acceptable standard. Responses in 
the category Retelling Main Events 
were also weak; only half the 
students achieved an acceptable 
standard. 


These results seem to indicate that 
students achieve at different levels 
when responding to what they 
have read, depending on the 
nature of interaction with the text 
demanded of the students. The two 
categories that had the highest 
percentages of students achieving 
an acceptable standard, Opinions 
and Predictions, are more reader- 
based. That is, success depends 
more on the knowledge and 
background the reader brings to 
the task and is not highly 
constrained by the text. 


The Story Structure and 
Characters categories each had 
55% of students achieving an 
acceptable standard. In order to 
answer these questions at an 
acceptable level, students are 


required to draw on their 


knowledge of the text itself and 
then organize the information. It 
seems that students find questions 
that depend on text-based 
knowledge more difficult than 
those that are primarily reader- 
based. 


The categories that elicited the 
poorest levels of achievement, 
Retelling Main Events and 
Personal Connections, are those 
that require the students to 
integrate both reader-based and 
text-based knowledge in order to 
make inferences and draw 
conclusions. 


It seems that students do draw on 
their prior knowledge and 
background experience when they 
read. However, they have more 
trouble focusing on the content of 
the text they are reading. They 
have the most difficulty making 
meaning by integrating 
information from their personal 
knowledge and the text. The 
results of the performance-based 
assessment suggest that helping 
students use their own experience 
as well as their knowledge of text 
features will help improve their 
reading comprehension. 


Writing Component 
General Description 

The writing component required 
students to write their own story, 
letter, or series of diary/journal 
entries using an idea that they 
might have generated while 
reading the book during the 
reading component. Students 
brainstormed ideas for writing in 
small groups with others who had 
read the same book. They were 
instructed not to retell the book 
they had read, simply to use an 
event, setting, character, or other 
story features as a springboard for 
their own writing. 


The writing component was scored 
using the same reporting 
categories and criteria as Part A: 
Writing of the Achievement Test. 
Provincial Results are shown in 
Table 2-10. 
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Table 2-10 : 
Grade Level 3 Language Learning Performance-Based Assessment 
Percentage Distribution of Scores by Reporting Category 


Writing Component 
June 1993 
Reporting Catego 
Be 9 gory 
Content and 
Standards Development Use of Language Conventions 

Meets or Exceeds the Standard of Excellence 37 39 47 

Approaches the Standard of Excellence 13.9 13.1 45.1 

Clearly Meets the Acceptable Standard 446 43.4 39.0 
“Approaches the Acceptable Standard. : 2B. kee ee) 
_teaty Below the Acceptable Standard _ Hea 85 


| INS (Insufficient) 


Examiner’s Observations 
Students achieving the acceptable 
standard but not the standard of 
excellence for Grade Level 3 
produced writing that followed a 
format with an identifiable 
beginning, events, and conclusion. 
Events and details were related 
and generally flowed from one to 
another. Endings were fairly 
predictable. Some specific 
vocabulary was used, although 
most words were general in nature. 
Familiar words were spelled 
correctly and unfamiliar words 
were spelled phonetically. End 
punctuation and capitalization 
were usually correct. 


Students achieving the standard of 
excellence for Grade Level 3 
produced writing that contained a 
direct relationship among events, 
providing for a smooth flow from 
one action to the next. Character 
motivation was consistent 
throughout, and most events 
contributed to the main character’s 
achievement of his/her goal. These 
writers showed an awareness of 
audience by providing details to 
explain why events occurred and 
by providing information about the 
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characters’ emotional responses to 
the events. Events were brought to 
an appropriate conclusion. The 
writer’s voice was often heard 
through the writing, captivating 
and holding the reader’s interest. 
The lengths and types of sentences 
varied. Specific adjectives and 
colourful expressions were found 
throughout the writing. End 
punctuation and capitalization 
were correct. Most words, familiar 
and unfamiliar, were spelled 
conventionally. 


Students below the acceptable 
standard for Grade Level 3 
produced writing that contained 
few events or actions. Beginnings 
were confusing, and subsequent 
events, did not appear to be 
related. Few details were included 
to explain the events, and endings 
were unconnected or non-existent. 
Words were general and did not 
provide sufficient information to 
the reader. There was little 
evidence that the writer 
understood the correct use of end 
punctuation and capitalization. 
Words were difficult to discern and 
were generally spelled 
phonetically. 


Relationship between the 
Performance-Based Assessment 
and the Achievement Test 
Correlations among all four 
components of the Grade Level 3 
Language Learning assessment 
(multiple-choice reading 
component of the achievement 
test, writing component of the 
achievement test, PBA reading 
component, and PBA writing 
component) were calculated. 
Positive relationships, with 
correlation coefficients ranging 
from .48 to .55, were found to exist 


. among all four components. The 


findings of this study are 
confirmed by other research 
studies where correlations average 
about .45. 


This finding is encouraging as it 
indicates that there is a positive 
relationship among the four types 
of Language Learning 
assessments. This would be 
expected if we are, in fact, 
assessing outcomes related to the 
same subject area. The fact that 
the correlation coefficients fall 
within a mid-range is also 
encouraging because it means that 
the performance-based assessment 


does not exactly duplicate 
information about student 
achievement provided by the 
machine-scorable component. 
Somewhat different aspects of 
achievement are being assessed. We 
are learning something different 
about what students know and can 
do from each of the two types of 
assessment. 


Performance-based assessment 
allows us to see more closely the 
kinds of knowledge students use 
when they are reading in situations 
similar to those encountered in the 
classroom on a daily basis. In 
responding to open-ended questions, 
students are able to select from a 
variety of strategies, depending on 
their own strengths as well as on 
the demands of the task itself. The 
examination and analysis of those 
responses and the strategies used 
give us more of a profile of the 
processes students use while 
reading. That information can then 
be used to outline more specifically 
students’ strengths and weaknesses 
in order to design programs that 
better meet their needs. 


Students did not do as well on the 
writing component of the 
performance-based assessment as 
they did on the writing component 
of the achievement test. Fewer 
students achieved an acceptable 
standard on the performance-based 
assessment in all three reporting 
categories. In Use of Language, 
fully 10% fewer students (60.4% as 
compared to 71.3%) met the 
acceptable standard on the 
performance-based assessment. 


Students had more difficulty with a 
writing task that was connected to 
a reading experience than they did 
on a task that was more open in 
nature. Again, it seems that the 
more students are required to 
integrate a variety of strategies in 
order to complete a task, the more 
difficulty they have. The writing 
component of the performance- 
based assessment required 
students not only to comprehend 
the story they read, but to 
integrate that with their own ideas 
to create an original story. In 
contrast, the writing component of 
the achievement test required 


students to look at a visual and then 
expand on their own ideas. Although 
reading and writing are often 
integrated in Grade Level 3 
classrooms, students have not yet 
mastered the application of a variety 
of strategies associated with reading 
and writing to a single task. 


These findings support the idea that 
a combination of assessment 
strategies (multiple-choice, 
performance-based assessments, 
and ongoing classroom-based 
assessments) are needed to provide 
a complete picture of student 
achievement in Language Learning. 


Context for Learning 
General Description 

In June 1993, a random selection of 
1961 Language Learning students 
in Grade 3 participated in a study 
that surveyed students’ attitudes on 
various language learning issues. 
The results from the survey are 
presented in Table 2-11. 


Table 2-11 


Grade Level 3 Language Learning 
Survey on Attitudes Toward Language Learning 
Percentage Distribution of Student Responses 


June 1993 
Statement Yes No Not Sure No Response 
1. I think reading is an important skill. 95.5 1.3 29 04 
2 Everyone needs to know how to read. 83.6 74 85 05 
3. After I've read something by myself, | like to: 
a _ Draw apicture 46.0 35.5 12.6 59 
b. Tell someone about it 47.0 33.1 12.7 7A 
c. _ Find another book like it 515 28.9 129 68 
d. Write something about it 35.0 433 13.1 86 
Continued 
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Table 2-11 (continued) 


ea 
Statement No Not Sure No Response 
4. | am a good reader. 56 18.8 1.4 
5. 1 am a good writer. 85 21.8 16 
6 Like writing: 
a Reports 345 13.0 6.1 
b. Jourals/diaries 32.7 13.4 69 
c. Letters 172 112 5.7 
d = Stories 83 75 42 
e. Poems 25.5 15.7 59 
Statement Sometimes Never No Response 
7. | keep my writing in a writing foider. 40.1 145 3.0 
8. Before | write, | like to: 
a Talk to my classmates 445 222 54 
b. Drawaweb 33.0 40.4 43 
c. Make achart 372 35.8 5.6 
d. _ Listen to my teacher 23.3 78 44 
e. imagine 249 94 3.1 
9. I like to: 
a Talk about books | have read 447 213 48 
b. Listen to my teacher read stories 18.5 40 3.1 
c. Read during my spare time 40.0 83 40 
d. Write during my spare time ~ 48.7 20.9 46 
10. | choose my books based on: 
a Author 38.5 32.7 46 
b. Topic (for example, mysteries) 28.8 10.4 441 
c. The pictures 38.0 29.3 51 
d. How hard the words are 35.3 25.9 45 
11. | use the school library to choose: 
a __ Stories for free reading 28.7 8.1 ' 3.7 
b. Information books (for example, animal books, 41.9 13.4 53 
books on amazing facts, or encyclopedias) 
c. Books to leam how to do things 467 - 121 45 
Continued 
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Table 2-11 (continued) 


Statement Often Sometimes Never No Response 
i 
12. This year, | read or used: 
a Magazines 27.0 39.8 278 55 
b. Newspapers 193 32.5 42.8 5.4 
c. Dictionaries 37.9 43.1 141 48 
d. Encyclopedias 23.0 39.6 32.3 5.0 
e. Atlas/Globe 273, 18 258 54 
f. Poetry 365 39.6 18.5 5.5 
g. _ Fiction books 59.9 26.7 87 47 
h. Information books (non-fiction) 42.6 43 10.4 5.8 
i. Comics 53.1 26.6 16.0 43 


Student Attitude 
Questionnaire 

Results indicate that of the 
students surveyed, 95.5% believe 
that reading is an important skill. 
Analysis of the student 
questionnaires indicate that the 
majority of students (85.6%) often 
or sometimes choose their reading 
material by topic and the most 
often read materials are fiction 
books, comics, and information 
books. Students use the school 
library mainly for choosing leisure 
reading materials. Students’ 
favoured types of writing are 
stories, letters, and poems; 74.4% of 
the surveyed students like to listen 
to their teacher read stories. 


The students surveyed tended to 
underestimate their abilities in 
writing and reading; 68.1% of these 
students thought they were good 
writers, and 74.2% thought that 
they were good readers. In actual 
fact, on the achievement test 

81.3% and 82.2% of students 
achieved the acceptable standard in 
writing and reading respectively. 


Relationship between Student 
Attitudes and the 
Achievement Test 

We were able to compare the 
results of 940 of the 1 961 students 


who participated in the attitude 
survey with their Grade Level 3 
Language Learning Achievement 
Test results. Of the 152 students 
who achieved the standard of 
excellence on the total achievement 
test, 90.1% believe that they are 
good readers. This is sharply 
contrasted by students who 
performed below the acceptable 
standard (n=132), where only 
44.9% believed themselves to be 
good readers. 


Students achieving the standard of 
excellence on the total test tend to 


‘be capable and enthusiastic 


readers. It was not surprising to 
find that 68.5% of those students 
achieving the standard of excellence 
read for enjoyment during their 
leisure time, whereas the same is 
true for only 46.2% of students who 
only achieved the acceptable 
standard and 40.8% of those 
performing below the acceptable 
standard. 


It is interesting to note that when 
students who are below the 
acceptable standard read in their 
spare time, they do so in order to 
learn something specific more often 
(40.2%) than do students who 
achieved the acceptable standard 
(38.8%) or who achieved the 


standard of excellence (30.2%). This 
reinforces the belief held by many 
teachers that reading, especially for 
weaker students, must have a real 
purpose in order for students to 
actively engage it. 


In preparation for writing, only one _ 
in four students surveyed like to talk 
to classmates, draw a web, or make 
a chart, whereas two out of three 
students in this group like to listen 
to their teacher and imagine. These 
activities are quite different in 
nature; the former three require the 
student to be actively involved with 
others or alone when producing a 
product, whereas the latter two are 
activities that allow students to be 
more passive. This suggests that 
students are more comfortable in 
situations where they are required 
to take less personal risk. 


Teacher Questionnaire 
In July 1993, 280 teachers from 
across the province, including 150 
teachers who were marking the 
writing component of the Grade 
Level 3 Achievement Test, 
participated in a study that 
surveyed the instructional strategies 
and learning environments they 
provided in the classroom. The 
results from the survey are 
presented in Table 2-12. The vast 
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majority of the teachers surveyed 
provide time for silent reading at 
least once per week. Teachers report 
that classroom environments are 
“print rich” with magazines, 


newspapers, both fiction and non- 
fiction books, as well as reference 
materials. Students who achieved 
the acceptable standard tend to 
have access to these resources 


slightly more often than do students 
who have not yet achieved the 
acceptable standard. It is interesting 
to note that all teachers surveyed set 
aside time to read to their students. 


Table 2-12 

Grade Level 3 Language Learning 
Teacher Survey on Instructional Practices and Learning Environments 
Percentage Distribution of Teacher Responses 


June 1993 
Less Than Several 
Once a Once a Times a No 
Statement Never Week Week . Week Every Day | Response 
1. Did your students write in L.L.? 0.0 18 75 46.1 425 21 
2 Did your students write in a journal for L.L.? 43 12.1 271 40.7 143 14 
3 Did your students use the schoo! Library? 04 0.7 618 32.1 39 11 
4. Did you set aside time to read to your students? 0.0 25 3.9 36.4 56.4 0.7 
5. Did your students do silent reading? 0.0 0.7 3.9 25.0 70.0 04 
es eee 
No 
Statement Never Rarely Sometimes Often Always Response 
6. Did your students have a choice about what they 0.0 0.7 214 59.6 17.5 07 
wrote? ; 
7. Did your students do prewriting activities before 0.0 32 218 439 30.4 0.7 
they wrote? 
8. Did your students use a computer at school to do 
the following? 
a _—_ Allof their writing (from draft to final copy) 63.6 15.7 10.7 18 0.7 75 
b. Some of their writing 468 23.2 2t4 6.1 04 25 
9. Did your students write a number of drafts before 46 23.9 33.6 22.5 11.4 39 
handing in the final copy? 
10. Did you have conferences with your students 11 13.9 368 346 9.6 39 
about their writing? 
11. Did you use the following instructional strategies 
with your students? 
a Reading workshop 20.7 13.2 22.9 23.9 75 118 
b. Paired Reading 29 11.8 28.2 45.4 86 32 
c. Learning Centres 12.9 243 29.3 23.6 46 54 
d Drama 46 39.3 37.1 13.6 11 43 
e. Group Work 0.7 32 23.6 579 125 21 
f. | Cooperative Leaming 43 11.4 22.9 B2 15.0 32 
Continued 
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Table 2-12 (continued) 


12. 


13. 


14. 


16. 


16. 


Statement No Yes No Response 

Did you use the following types of assessments with your 

students? 

a Multiple-choice test 243 70.0 5.7 

b. Short-answer test 11.1 82.9 6.1 

c.  True-false test 493 43.6 7.1 

d Interpretation of charts, diagrams, pictures, 18 91.8 64 

graphs, and maps. 
e. Creation of a diagram, picture, mural, collage, or 3.9 90.0 6.1 
model to show understanding 

f. Oral presentation 39 90.7 54 

g. Arrangement of events in order 68 87.5 5.7 

h.___ Classification of picturesAvords/phrases 89 843 68 

Were the following materials available in the classroom 

for the students? 

a Magazines 96 846 5.7 

b. Newspapers 50.4 42.1 75 

c. Dictionaries 07 ~ 943 50 

d. Encyclopedias 41.4 50.7 79 

e.  Atlas/Giobe 7A 87.9 5.0 

f. Poetry 32 91.1 5.7 

g. Fiction books 0.0 95.0 5.0 

h. —_ Non-fiction books 1 93.6 54 

Are the following reading series available in your school 

for you to use? 

a Early Bird Collection 679 6.1 26.1 

b. Impressions 182 70.0 11.8 

c. Joumeys 454 32.5 22.1 

ad Networks 37 436 20.7 

e. Unicorn 59.3 118 28.9 

f. Waves 668 3.6 29.6 

g. Reflections 65.0 5.0 30.0 

Did your students, as a class, experience the following? 

a _Live theater 

b. Concert 21.4 718 68 

c. Symphony/Opera 243 67.1 86 

d Movie 62.1 26.8 114 

e. Museum/Art Gallery 96 84.6 5.7 
346 56.8 86 

Did your students have a portfolio or writing folder for 

samples of their writing? 89 85.4 5.7 
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Slightly more than one in four . 
teachers surveyed said that their 
students use the computer for 
some of their writing at least once 
per week. It is an encouraging sign 
that the technology is becoming 
more readily accessible in the 
lower elementary grades; as it 
becomes more available, we should 
see an emerging trend of younger 
students actively using the 
technology for more of their work. 


Of the teachers surveyed, one in 
four did not respond to the 
question about the reading 
resources in their schools. They 
appear to be uncertain about the 
availability of the specific 
resources mentioned in this 
survey. 


Relationship between 
Learning Environment and 
the Achievement Test 
Achievement test results of 509 
students were matched with 
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teachers who participated in the 
survey of instructional practices 
and learning environments. 
Students were split into three 
groups: students achieving the 
standard of excellence, students 
achieving the acceptable standard 
but not the standard of excellence, 
and students not yet achieving the 
acceptable standard. Differences in 
reported teaching behaviours 
matched to these students were 
examined. All of the teachers in 
this matched study reported that 
their students used their school 
library at least once per week. All 
teachers provided field trip 
experiences outside the regular 
classroom milieu: 92.8% of 
teachers of students who achieved 
the standard of excellence provided 
an opportunity for viewing live 
theatre, and 100% of these 
teachers took their students to art 
galleries and museums. 


The study showed that 25.5% of 
teachers of students achieving the 
standard of excellence always have 
their students write multiple 
drafts of compositions, compared to 
18.0% of teachers of students 
achieving the acceptable standard 
and 15.3% of teachers of students 
who were below the acceptable 
standard. 


Furthermore, 7.2% of the teachers 
of students who achieved the 
standard of excellence always 
conference with their students 
about their writing, whereas 6.8% 
of teachers of students achieving 
the acceptable standard and only 
4.7% of those teachers of students 
who were below the acceptable 
standard follow the same practice. 


_ Section 3 


Social Studies. 


In 1993 a number of assessments 
were carried out in Grade 6 Social 
Studies. The achievement test was 
administered to students province- 
wide. Student self-reports and 
teacher ratings were collected in a 
sample of classrooms as a means of 
gathering information about 
process skills. As well, a sample of 
Grade 6 Social Studies students 
and teachers completed 
questionnaires designed to 
examine the relationships among 
various contexts for learning and 
achievement. The results from all 
these assessments follow. 


Achievement Test 
General Description 


The Grade 6 Social Studies 
Achievement Test had two parts. 
Part A consisted of 50 multiple- 
choice questions and Part B was a 
writing assignment. Part A was 
worth 70% and Part B was worth 
30% of the total test mark. 
Students had 60 minutes to 
complete each part of the test. 


Statistics for the total test and for 
the components are based on the 
results achieved by 32 630 
students: 29 802 wrote the regular 
form and 2 828 participated in the 
achievement-over-time study, as 
shown in Table 3-3. This section of 
the report provides answers to the 
following questions: 


* How many Grade 6 students 
wrote each form of the test or 
were absent or exempt? 


e What percentage of students who 
wrote the Grade 6 Social Studies 
Achievement Test achieved the 
acceptable standard? 


* What percentage of students who 
wrote the Grade 6 Social Studies 
Achievement Test achieved the 
standard of excellence ? 


* What did Grade 6 students know 
and what could they do in Social 
Studies? 


* What parts of the Social Studies 
curriculum were difficult for 
Grade 6 students? 


Summary of Results 
Results show that 80.8% of the 
students who wrote the test 
achieved the acceptable standard 
and 8.0% achieved the standard of 
excellence. These results were 
lower than expected. 


Content of ‘the Test 


Of the 50 multiple-choice questions 
in Part A, 19 questions on Topic A, 
which dealt with Local 
Government, 15 were on Topic B, 
which dealt with Greece: An 
Ancient Civilization, and 16 were 
on Topic C, which dealt with 
China: A Pacific Rim Nation. 


The Part B writing assignment 
asked students to write one to 
three paragraphs on the issue 
“Should governments help people 
meet their needs?” 


The test assessed students’ 
knowledge of facts, concepts, and 
generalizations, as well as their 
application of process skills such as 
using information to draw 
conclusions. 
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Test Blueprint 


The test blueprint for Part A: (topic) being assessed and 
Muitiple Choice shows the according to the knowledge and 
distribution of questions according process skills required to answer 
to the curricular content area the question. 


Table 3-1 
Grade 6 Social Studies 
Achievement Test Blueprint 
Part A: Multiple Choice 
June 1993 
Question Distribution 
Topic B Topic C 
Topic A Greece: An Ancient | China: A Pacific Rim | Proportion of 
Knowledge/Skills Local Government Civilization Nation Total Score 
E) Understands generalizations, 1, 2,3, 18 4, 9, 10, 11, 8, 16, 30, 31, 
® concepts, related facts and 22, 36, 37, 39, 49 17, 20, 21, 25, 27 32, 33, 34, 35 36% 
FY content 
° 
¢ 
x 
a Locating, organizing, interpreting | 40, 42, 46, 47, 48 7, 26, 44 13, 14, 28, 29, 
= information* , 43, 45 
o 
o ; . 3% 
4 Analyzing, synthesizing, 19, 23, 38, 41, 50 5,6, 24 12, 15 
8 evaluating 
a 
Proportion of Total Score 27% 21% 22% 70% 


* This includes the section in the Program of Studies entitled “Geography/Mapping” under Process Skills. 
Three questions were deleted because the choices were ambiguously worded (22, 33, 45). 
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The blueprint for Part B: Written 
Response shows the reporting 
categories and the proportion of 
score. 


Table 3-2 


Grade 6 Social Studies 
Achievement Test Blueprint 
-Part B: Written Response 
dune 1993 
Description of Writing Activity Reporting Category Proportion of Score 
Express an opinion/position using 1. State and Support an Opinion/Position 
examples for support. Students are expected to persuasively give their 
opinion/position and use relevant examples to 20% 
support it. 


2 Quality of Language and Expression 
Students are expected to express their ideas in an 
organized manner with clear, effective language, 
using correct conventions of spelling, punctuation, 
and grammar. 


10% 
TOTAL 30% 


‘Oo 


ee 


eee 
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Student Participation Achievement Test, who were Immersion students are reported 


In June 1998, principals reported a absent, or who were exempted. In directly to participating schools 
total population of 39 097 students total, 89.7% of the students and jurisdictions and are not 

in Grade 6. Table 3-3 presents the reported to be in Grade 6 in June _ included in the calculation of the 
number and percentage 1993 wrote the Grade 6 Social provincial results presented in this 
distribution of students who wrote Studies Achievement Test. Results section of the report. 

the Grade 6 Social Studies for Francophone and French 


Table 3-3 
Grade 6 Social Studies 
Student Participation 
June 1993 
Number Percentage 
been of Students of Students 
eo 
29 802° 
2 828 
964 
932. 
; : 9852 
Other Students Who Wrote: 2 432 
Francophone—Taught in French, Wrote in French 0 
Francophone—Taught in French, Wrote in English 176 
French Immersion—Taught in French, Wrote in English 24 
French immersion—Taught in French, Wrote in French 2 232 
Students Absent , 1 709 3.2 
Students Exempted from Writing 2 326 14.7 
Categories of Exemption: 
1. Special Needs Students 1072 2.7 
2. Subject Was Not Taught This Term 508 13 
3. Students in English as a Second Language 407 1.0 
4. Language of Instruction Was Not English 63 0.2 
5. Other (as approved by the Superintendent) 276 0.7 
Total Principals’ Reported Population 
Test Day, June 1993: 39 097 - 100.0 
Grade 6 Enrollment: September 30, 1992 38 484 


*Provincial results are based on test scores achieved by students who were instructed in English and who wrote the English form of the 
achievement test, and those students who were instructed in a language other than English or French and who wrote the English form of 
the test. They are indicated by the shaded area of the tabie. 
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Results in Relation 
to Standards 


Achievement test results are 
reported in relation to assessment 
and achievement standards. The 
Provincial Assessment 
Standard is the lowest score on a 
test that a student must achieve for 
his/her performance to be judged 
“acceptable” and/or “excellent”. The 
Provincial Achievement 
Standard refers to the percentage 
of students writing the test who are 
expected to achieve the Provincial 
Assessment Standard. 


Figures 3-1 and 3-2 and Table 3-4 
show the percentage of students 
achieving provincial assessment 
standards in relation to provincial 
achievement standards on the total 
test and on components of the test. 
These data are based on the results 
of the 32 630 students in regular 
programs who wrote the test. They 
show that the percentage of 
students achieving the standard of 
excellence on the knowledge 
questions was considerably higher 
than expected. Fewer students than 
expected achieved all other 
standards set for the Grade 6 Social 
Studies Achievement Test. 


When interpreting these findings, it 
is important to remember that they 
are based on the test scores of only 
a portion (83.5%) of the students 
reported to have been enrolled in 
Grade 6 in June 1993, i.e., those 
students in regular programs who 
wrote the achievement test. These 
figures tell us nothing about the 
levels of achievement of students in 
Francophone and French 
Immersion programs, of students 
who were absent on the day of 
writing, or of those students who 
were exempted from writing. 


Figure 3-1 
Grade 6 Social Studies 


Percentage of Students Achieving Acceptable Standard on the 
Total Test and on Components of the Test 


June 1993 


100 
80 
60 


Knowledge 


Process Skills 


Total Test 


* Provincial Achievement Standard—85% of the students who wrote the test are expected to 


achieve the acceptable standard 


Figure 3-2 
Grade 6 Social Studies 


Percentage of Students Achieving Standard of Excellence on the 
Total Test and on Components of the Test 


June 1993 


Knowledge 


Total Test 


*Provincial Achievement Standard—15% of the students who wrote the test are expected to 


achieve the standard of excellence 


Earlier studies on the language of 
testing show that the standards 
developed for the English version 
of the test cannot be directly 
applied to French Immersion 
students writing the translation; 
standards for the French 


Immersion and Francophone 
students were not set separately 
due to the small numbers involved. 
Thus, these students are not 
included in any discussion of the 
numbers of students achieving 
standards. 
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Table 3-4 
Grade 6 Social Studies 


Students Achieving Standards on the Total Test and on Components of the Test 
June 1993 


Reporting Category 
Standard of Excellence 
Total Test 

Skills Only 

Knowledge Only 


Acceptable Standard 
Total Test 

Skills Only 

Knowledge Only 


Below Acceptable Standard 
on Both Components 


Students Achieving Assessment 
Standard 


Possible 
Score 


“The Provincial Assessment Standard is a score determined by standard-setting procedures and is the lowest score a student must 
achieve for his/her performance to be judged “acceptable” and/or “excellent” in relation to curricular expectations. See Appendix A. 


The Provincial Achievement Standard refers to the percentage of students expected to achieve the Provincial Assessment Standard. 


It should be noted that the actual 
percentages of students achieving 
standards on the total test for 
Grade 6 Social Studies (8.0% and 
80.8%) are based on 32 630 
students in the regular programs 
who wrote the test. 


If the percentages are based on the 
total June population reported by 
principals (not including the 
French Immersion and 
Francophone students who wrote 
the test), the percentages achieving 
standards would be: 

7.1% achieving the standard of 

excellence, 

71.9% achieving the acceptable 

standard. 
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If the percentages are based on the 
September 30, 1992, Grade 6 
enrollment (36 052) not including 
the Francophone and French 
Immersion students who wrote the 
test, the percentages achieving 
standards would be: 

7.3% achieving the standard of 

excellence, 

73.1% achieving the acceptable 

standard. 


It is emphasized that the 
percentages based on enrollment 
present the lowest estimate of 
achievement. It is highly likely 
that some of the students who were 
absent, exempt, or not accounted 
for could have achieved standards. 


The number of students achieving 
the acceptable standard and the 
standard of excellence for each 
school was analyzed to determine 
whether schools were below, 
meeting, or exceeding provincial 
achievement standards. Schools 
classified as meeting provincial 
achievement standards were those 
for which the difference between 
the actual number of students and 
the expected number of students at 
or above standards was not 


statistically significant. Differences 
are only reported when there is a 
5% or smaller probability that a 
difference of that size could occur 
by chance. The results are reported 
in Table 3-5. 


Table 3-5 shows the 81.2% of 
schools were considered to have 
achieved the provincial 
achievement standard for the total 
test at the standard of excellence. 
About one-quarter of schools were 


significantly below the provincial 
achievement standard. School 
administrators and teachers 


_ should critically examine social 


studies achievement test results to 
determine what they are going to 
do to improve the level of student 
achievement. Although this is true 
for all schools, it is particularly 
important for those schools 
performing below the provincial 
achievement standard. 


Table 3-5 
Grade 6 Social Studies 
Percentage Distribution of Schools* Meeting Provincial Achievement Standards on the Total Test 
(N = 993) 
June 1993 
Percentage Distribution of Schools 
Significantly Below Not Significantly Different Significantly Above 
Provincial Achievement From Provincial Provincial Achievement 
Standard Standard Achievement Standard Standard 
Standard of Excellence 18.8 78.7 2.5. 
Acceptable Standard 26.4 67.5 6.1 


*Schools with fewer than five students are excluded, as the statistical significance of the difference between the number actually meeting the 
standard and the number expected to meet the standard when calculated and reported is not educationally meaningful. 


Results for Part A: 
Individual Multiple- 
Choice Questions 

Tables 3-6A to 3-6C show the 
percentage of students who chose 
each alternative (A, B, C, and D) 
for each multiple-choice question. 


The results shown in these tables 
can best be used in conjunction 
with the tables in jurisdiction and 


school reports. Variations in 
patterns of students’ responses to 
questions can help to indicate 
strengths and weaknesses in local 
educational programs. 


Statistics represented in these 
tables are based on results 
achieved by 30 763 students (those 
who wrote the regular form and 
those who wrote Form F). 


The sum of the percentages for 
each question may be less than 
100% because the No Response 
category is not reported in these 
tables. 


For a discussion of individual 
multiple-choice questions, see 
Examiner’s Observations, pages 38 
to 40. 
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Table 3-6A 


Grade 6 Social Studies 
Results for Individual Multiple-Choice Questions 
Topic A—Local Government 


June 1993 
fe > - 
Distribution of Responses (%) 
Item Component Category Curriculum Standard 
A B Cc D 
1 8.2 17.2 72.5* 2.1 Knowledge Cc know examples of physical needs 
2 30.0 39.3* 9.6 20.7 Knowledge Cc know examples of social needs 
3 76.1* 3.7 10.3 9.8 Knowledge Cc know examples of government services 
36 14.1 10.5 60.2* 15.1 Knowledge Cc understand the right of free speech 
18 5.2 9.0 76.9* 8.8 Knowledge G understand the democratic process 
37 6.6 10.5 8.2 74.6" Knowledge G understand the responsibilities of a citizen 
in a democracy 
39 17.3 7.1 1.4 74.1* Knowledge G identify a lobby group activity 
49 12.7 16.7 30.0* 40.2* Knowledge G understand appropriate lobby group activity 
40 12.0 61.2" 9.2 17.1 Process Skills L compare information on a graph to a chart 
42 18.3 21.1 51.6" 8.9 Process Skills L interpret information to distinguish 
fact from opinion 
46 11.5 57.1" 28.3 2.8 Process Skills L use scale to calculate distance 
47 22.9 17.7 35.4" 23.6 Process Skills L interpret maps to discover relationships 
of scale 
48 87.5" 5.6 3.7 2.8 Process Skills L select an efficient means of communicating 
information 
19 aT 11.3 21.7 19.3 Process Skills A draw a conclusion about democracy 
23 72.3" 18.4 3.4 6.0 Process Skills A draw a conclusion based on given information 
38 11.4 11.7 70.9* 5.7 Process Skills A analyze the result of actions in a democracy 
41 25.3 36.5* 17.0 20.9 Process Skills A judge how useful the data in a chart would be 
for different purposes 
50 13.2 26.4 30.6" 29.3 Process Skills A evaluate sources of information 


“correct answer (Item 49 has two correct answers) 


Category Legend: F—Facts 
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C—Concepts 


G—Generalizations 


L—Locating, Organizing, Interpreting Information 
A—Analyzing, Synthesizing, Evaluating 


Continued 


Table 3-6B 


Grade 6 Social Studies 


Results for Individual Multiple-Choice Questions 
Topic B—Greece: An Ancient Civilization 


June 1993 
Distribution of Responses (%) 
Item hg ae Component Category Curriculum Standard 
A B Cc D 
4 7.1 14.2 5.0 73.5" Knowledge F understand psychological needs 
9 53.9" 14.1 25.2" 6.7 Knowledge | F identify a psychological need in Ancient 
; Greece 
10 2.5 46 1.2 91.7* Knowledge F understand the role of slaves in Ancient 
| Greece 
11 56.6" 7.9 14.3 21.2 Knowledge F know how Ancient Greeks met some 
physical needs 
17 13.9 10.7 10.6 64.7" Knowledge F understand the role of the Assembly in 
Ancient Greece 
20 8.3 80.7* 5.7 5.0 Knowledge F understand that democracy is common to 
Canada and Ancient Greece 
27 93.6* 2.4 1.8 2.2 Knowledge F understand the role of slaves in 
| Ancient Greece 
aan a 
21 11.2 59.4* 17.4 11.9 Knowledge G understand the purpose of government 
25 10.6 20.5 66.0* 2.7 Knowledge G understand class structure in Ancient Greece 
oe = : 
7 36.6" 19.0 33.1 11.4 Process Skills L interpret comments to identify an opinion 
26 25.6 32.6 33.6* 8.0 Process Skills L interpret ‘comments to identify a fact 
44 22.6 25.4 20.3 31.3* Process Skills L use latitude and longitude to locate 
geographical places 
5 8.3 16.7 6.0 68.7* Process Skills A synthesize information to select the 
best conclusion 
6 8.7 77.6* 4.9 8.7 Process Skills A draw a conclusion based on various 
comments 
24 5.3 86.4* 6.8 1.4 Process Skills A evaluate statements of opinion 


*correct answer (Item 9 has two correct answers) 


Category Legend: F—Facts 


C—Concepts 


G—Generalizations 


L—Locating, Organizing, Interpreting Information 


A—Analyzing, Synthesizing, Evaluating 


Continued 
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Table 3-6C 
Grade 6 Social Studies 


Results for Individual Multiple-Choice Questions 


Topic C—China: A Pacific Rim Nation 


June 1993 
Distribution of Responses (%) 
Item Component Category Curriculum Standard 
A B Cc D 
8 12.1 3.1 81.8* 2.9 Knowledge F understand role of grandparents in meeting 

social needs in China 

16 19.3 24.4 7.7 48.5* Knowledge F understand the importance of terracing 
in China’s farming 

30 5.5 15.0 5.3 74.0* Knowledge F know why Canada - China twinning 
is important 

31 41 55.4* 33.3 7.0 Knowledge F understand why China is “opening to 
the world” 

32 24.4 8.3 62.1* 5.0 Knowledge F understand the importance of the Pacific Rim 

34 7.9 9.4 2441 58.4* Knowledge “F know which province borders on the Pacific . 
Ocean : 

35 82.9" 3.8 4.2 9.0 Knowledge F know that silk originated in China 

13 7.3 47.0* 43 41.3 Process Skills L interpret comments about technology 

14 23.9 17.7 13.0 45.3* Process Skills L interpret Chinese comments about Canada 

28 14.0 71.0* 2.8 12.1 Process Skills L interpret a chart to select main idea 

29 10.9 7A 79.5* 2.4 Process Skills L interpret a chart to apply its information 

43 54.6* 15.8 16.7 12.6 Process Skills L tocate a Pacific Rim country on a map 

12 12.7 6.2 71.3" 9.7 Process Skills A evaluate information from different speakers’ 
comments 

15 54.8* 27.2 5.2 12.7 Process Skills A evaluate statements about Chinese policy 


*correct answer 


Category Legend: F—Facts 
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C—Concepts 


G—Generalizations 


L—Locating, Organizing, Interpreting Information 
A—Analyzing, Synthesizing, Evaluating 


Multiple-Choice 
Reporting Categories 
Table 3-7 shows the total marks 
possible and the raw score 
averages for the multiple-choice 
reporting categories. 


It is important to stress that the 


averages in the various reporting 
categories cannot be directly 


Table 3-7 


compared with one another. 
Rather, the results shown in Table 
3-7 can best be used in conjunction 
with parallel tables in the 
jurisdiction and school reports. 
Variations in patterns of results 
can help indicate strengths and 
weaknesses in local educational 


programs. 


Statistics presented in this table 
are based on results achieved by 
30 763 students (those who wrote 
the regular form and those who 
wrote Form F of the test). 


Grade 6 Social Studies 

Raw Score Results by Multiple-Choice Reporting Category 

June 1993 

Maximum Raw Score 
Possible Raw Score Standard 
Reporting Category Raw Score Average Deviation 
een 

Topic A— Local Government 18 109 — 3.2 
Topic B—Greece: An Ancient Civilization 15, 10.0 2.5 
Topic C—China: A Pacific Rim Nation 14 8.9 2.8 
Knowledge: (Facts, Concepts, Generalizations) 
Topic A 8 5.4 1.8 
Topic B 6.7 1.6 
Topic C 4.6 1.7 
Process Skills: 
Locating/Organizing/Interpreting 13 6.9 2.4 
Analyzing/Synthesizing/Evaluating 10 6.2 2.0 
Major Components: 
Knowledge 24 16.7 41 
Process Skills 23 13.1 3.8 
Total Muitiple Choice 47 29.8 7.3 | 
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Results for 
Part B: Written Response 


The number and percentage 
distribution of scores, by written- 
response reporting category, are 
presented in Table 3-8. These data 
are based on the results achieved 
by all 32 630 students who wrote 
English forms of the 1993 Grade 6 
Social Studies Achievement Test. 
The average weighted score on the 
written-response portion of the test 
was 16.5, with a standard 
deviation of 4.9. 


The percentage of students 
achieving standards on “State and 
Support an Opinion/Position” was 
only 58.4% compared to 41.6% who 
failed to achieve the acceptable 


standard; 72.7% of students 
achieved standards on “Quality of 
Language and Expression” 
compared to 27.4% of students who 
did not achieve standards. See 
page 41 for the Examiner’s 
Observations on the written 
response. 


Although the papers were scored 
on a one-marker system, randomly 
selected papers were re-marked so 
that a second set of scores was 
available to confirm scoring 
consistency. Of the scores awarded 
on the second reading, 42.4% were 
identical to the scale scores 
originally awarded and 46.5% 
varied by only one point. The one- 
marker system produces results 


that are reasonably reliable for 
groups of 25 or more students. 
Achievement test scores, however, 
are less reliable for individual 
students. 


It is possible to draw conclusions 
about program strengths and 
weaknesses by comparing local 
percentage distributions of scale 
points in each reporting category 
with provincial distributions. The 
comparisons are most clearly 
understood in the context of the 
assignment students responded to 
and the scoring guides used to grade 
their work. Copies of the writing 
assignment and scoring guides are 
available from Student Evaluation 
Branch, Alberta Education. 


Table 3-8 

Grade 6 Social Studies 
Number and Percentage Distribution of Scores by Written-R 
June 1993 


Score Scale Points 


5 
(Excellent) 


4 
(Good) 


3 
(Acceptable) 


img oe 
eee © (Insufficient) “ é 


State and Support an Opinion/Position 
Number 


ghee: 


Quality of Language and Expression 


17 976 


esponse Reporting Category 


Number Percent 


970 3.0 
4764 146 
55.1 


Examiner’s Observations 
Achievement in Grade 6 Social 
Studies is slightly better than 
1989. However, not enough 
students achieved either the 
acceptable standard or the 
standard of excellence on the total 
test. 
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Multiple Choice 

Students found the process skills 
questions on the multiple-choice 
section of the test very challenging. 
Sample questions from the test and 
accompanying discussion are 
provided to highlight the strengths 
and weaknesses of students who 


achieved the acceptable standard 
and the standard of excellence. For 
each sample question, the correct 
answer is marked with an asterisk. 
The percentage of students 
choosing each alternative is also 
provided. 


Use the following information to answer question 37. 


When the people of Big Lake discussed the need for a swimming 
pool, some supported the idea while others were against it. The 
town council voted in favour of building the swimming pool. 


“This decision 
upsets me. My taxes will 
increase to pay for this poo} 
and I don’t even know 


Mrs. Jones 


37. If the town council does not change its decision, Mrs. Jones has a responsibility to 


6.6 A. learn to swim 
10.5 B. refuse to pay her taxes 
8.2 C. refuse to use the new pool 
74.6 *D. accept the council's decision even if she is upset 


Use the following information to answer question 15. 


“The youth today want 
too much: new clothes, cars, and 

freedom. The old ways were better 
because they emphasized the important 
things in life, such as family 
and nature.” 


¢ “Visitors from other countries 

are good for China. They bring money 
and new ideas. To improve my English 
I bought a radio so that I could 
listen to English-speaking 


“Trade with western 
countries is very important to the 
Chinese economy. We have many products 
that countries like Canada and the United States 
need and want. These countries are willing to 
pay a great deal of money for them, 
which can only benefit the 
Chinese economy.” 


“It is the opinion of our 

government that foreign visitors, goods, 

and money influence our youth. Young 
people must learn the benefits of life 

: in China and of living in 

a Chinese culture.” 


Minister of Culture Businessman 


15. Sin would most likely agree with which statement? 


54.8 *A. The government should strongly promote Chinese values among its young people 


27.2 B. The government should encourage bringing cultural groups into China. 
5.2 C. Chinese people should be allowed to buy whatever they like. 
12.7 D. China should import more products to change young people 


. Acceptable Standard 


For question 37, students needed 
to have a basic understanding of 
rights and responsibilities. 
Students achieving the acceptable 
standard have a good 
understanding of the necessary 
balance between rights and 
responsibilities in local 
government issues. 


For question 15, students were 
required to analyze information 
and draw an inference. Results 
indicated that students who 
achieved the acceptable standard 
but not the standard of excellence 
had difficulty making inferences. 
However, students achieving the 
standard of excellence were clearly 
able to do so. 


The strengths of students who 
achieved the acceptable standard 
include an ability to: 

* apply their knowledge of basic 
concepts such as human needs, 
class structure, rights, 
responsibilities, lobby groups, 
and sharing 

° synthesize information to draw 
simple conclusions 

® compare and contrast ways 
different societies meet needs. 


Many students, however, did not 
do as well as expected in: 


° recalling some basic knowledge 


as outlined in the course of 
study, such as knowledge about 
Pacific Rim and government 

*® analyzing and synthesizing 
information to make inferences 
and determine the main idea 

* comparing information, charts, 
and graphs 

° distinguishing between fact and 
opinion 

* comparing maps of different 
scales and computing distances. 


39 


11. The picture below that shows what the Ancient Greeks would have used to satisfy a 


physical need 
56.6 ‘A. 79 ~=«=B. 


Sheaves of wheat Abacus beads 


Musical instrument 


Use the following information to answer question 41. 


The people of Brownsville are concerned about the increase in bicycling 
accidents over the last two years, as shown in the following chart and graph: 


Accidents Involving Bicycles 


Number of Accidents 
Type of Accidents 1991 1992 
Automobile and bicycle accidents 13 17 
Pedestrian and bicycle accidents 2 3 
Individual bicycle accidents 10 31 
Property damaged by bicycles 9 a 


Accidents Invoiving Bicycles 


Number of 2 
Accidents 


1991 1992 1991 1992 1991 1992 1991 1992 
Automobile Pedestrian Individual Property 


Type of Accidents 


Note to the reader: Pedestrian bars were intentionally reversed for another test question that 


41. 


probed the display of data 


A group who could best use the statistics from the chart would be 


25.3 A. hospital managers requesting increased funding 
36.5 *B. citizens who support a bylaw to restrict bicyclists 
17.0 C. citizens who oppose a bylaw to restrict bicyclists 
20.9 D. police officers when they charge bicycle offenders 
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Standard of Excellence 


For question 11, students had to 
have a basic understanding of 
human needs. A high percentage of 
students achieving the standard of 
excellence were able to identify a 
physical need. However, only 
slightly more than half of students 
achieving the acceptable standard 
were able to identify how the 
ancient Greeks satisfied a physical 
need. 


For question 41, students needed 
to make inferences by analyzing 
and comparing information 
contained in a chart and a graph. 
Many students who achieved the 
acceptable standard and some 
students who achieved the 
standard of excellence had 
problems selecting the correct 
inference. Too many students in 
both groups were drawn to the first 
alternative. 


Students who achieved the 

standard of excellence were able to: 

® clearly understand concepts as 
outlined in the course of study 

° select main ideas by reading 
information 

¢ analyze and synthesize 
information to make simple 
inferences and draw conclusions 

* identify points of view 

® compute distances on maps. 


However, some of these students 

did not do as well as expected in: 

® distinguishing between fact and 
opinion 

® making an inference by 
comparing information from a 
chart and a graph (see question 
41) 

* comparing maps of different 
scales. 


Written Response 

Although the quality of writing in 
social studies is slightly better in 
1993 than it was in 1989, not 
enough students achieved the 
acceptable standard. Teachers 
were disappointed with many of 
the papers that they marked. The 
markers’ specific observations 
include the following: 


* too few students handled the 
composition adequately 

e many students had difficulty 
organizing ideas 

* writing was general rather than 
specific and some students used 
unrelated personal examples 

® some students showed a lack of 
understanding of government 
and some confused the levels of 
government 

* many students had difficulty 
communicating ideas in support 
of their position 

* a small number of excellent 
papers were written 


A majority of teachers indicated 
that the writing task may have 
been too general and, rather than 
delimiting the topic, many 
students responded with broad 
statements that did not 
communicate clearly or effectively. 
These concerns will be addressed 
in future designs for written- 
response questions. 


Issues 

During the 1993 Grade 6 Social 
Studies marking session, teachers 
were asked to comment on the 
writing assignment. The markers 
made a number of suggestions 
regarding students’ writing in 
support of a position. For example, 
they indicated that taking a 
position on an issue should follow 
the generation of ideas and 
consideration of alternatives. 


Qualified positions can be 
supported as well as “for” and 
“against” positions. Students in 
Grade 6 classrooms can best 
address issues when they are 
specific and in familiar contexts. 


Performance-Based 
Assessment 


General Description 

A total of 79 Grade 6 teachers and 
1 809 students from a sample of 
schools throughout the province 
were involved in an assessment of 
process skills. Student and teacher 
survey instruments were ~ 
constructed to help assess the 
extent to which the identified 
processes have been acquired by 
students. The results are presented 
in Tables 3-9 (students) and 3-10 
(teachers). 


Students were asked to complete a 
self-assessment of 11 process skills 
areas. Teachers used three 
descriptors to rate each of their 
students on the same 11 processes 
areas. 


This assessment was intended to 
help describe the relationship 


between student results on the 


1993 Grade 6 Achievement Test 
and student acquisition of process 


_ skills. 


Examiner’s Observations 
The instruments used by the 
teachers and the students provided 
a limited amount of information 
regarding the identified process 
skills. However, many of the Grade 
6 teachers who participated in the 
study were pleased with the 
initiative and enthusiastic about 
the general trend to assess a 
broader range of skills. Some 
teachers indicated that the study 
should have been based on the 
skills as defined in the Program of 
Studies. 


Results show that students rate 
themselves highest in the 
processes of listening to others, 
persistence, questioning and 


’ problem posing, and 


inquisitiveness. Teachers rated 
students highest in the processes of 
questioning and problem posing, 
precision of language and thought, 
metacognition, and inquisitiveness. 
The ability to deal with several 
sources at one time and to consider 
alternative points of view 
(flexibility in thinking) was the 
area rated lowest by both students 
and teachers. The results from this 
assessment will be used as a basis 
to design future performance 
assessments that reflect broad 
learner expectations in the social 
studies program. 


Relationship between 
Performance-Based 
Assessment and the 
Achievement Test 
Correlations between teacher 
ratings of students’ process skills 
and the achievement test scores 
were calculated. Generally, there is 
a positive relationship between a 
student’s reported attainment of 
the identified process skills and 
his/her performance on the 
achievement test. For example, 
teacher ratings of student 
metacognitive skills correlated 0.51 
with student achievement test 
scores. The relationship between 
questioning and problem posing 
skills and achievement test scores 
was correlated at 0.44. 
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Table 3-9 

Grade 6 Social Studies 

Student Survey on Performance-Based Assessment 
Percentage Distribution of Student Responses 
June 1993 


Descriptors For Rating Scale 


Often: | consistently demonstrate this attribute 
Sometimes: | usually demonstrate this attribute, but not with consistency 
Not Yet: | rarely, if ever, demonstrate this attribute 


Rating Scale 
Attributes r . No 
Often | Sometimes ot Yet; Response 
| p 
1. Continuing to work when the solution to a problem is not 
easily found 
a. | stay on task but may take a break 29.5 64.8 5.0 0.7 
b. | seek other sources of information when | need help 39.5 55.2 4.8 0.5 
c. | do not need to be reminded to complete tasks/projects 33.4 53.9 11.9 0.8 
2. Thinking before acting 
a. | listen and understand directions before starting 53.9 42.7 2.7 0.7 
b. | listen to what other students say and do not ask unnecessary 
or the same questions 48.9 41.2 9.1 0.8 
c. | ask questions to make the assignment clear to me 53.5 40.3 5.6 0.6 
d. | study the problems and develop a plan to solve them 32.9 57.0 9.4 | . 0.7 
e. | think before answering 63.2 33.3 2.6 0.9 
3. Listens with understanding and feeling 
a. | repeat other students’ ideas in my own words 19.5 1 18.7 0.7 
b. | use body language to respond 25.3 51.4 22.6 0.6 
c. | ask questions that are on the topic 61.8 33.6 3.8 0.8 
d. | answer with acceptable actions or words 48.5 45.9 45 - 1.0 
4. Awareness of our own thinking 
a. When faced with a problem, | can explain what the problem is 33.7 59.8 5.7 0.8 
b. | can explain the parts of the problem that | already know 62.8 33.8 2.4 0.9 
c. | can identify the parts of the problem that | need to know 45.9 48.8 4.2 1.1 
d. | develop and carry out plans to solve problems 34.3 57.3 7.3 1.1 
e. | change my plan when it needs to be changed 54.1 39.5 5.3 1.0 
f. | think about what | did to solve the problem 36.6 49.4 13.2 0.8 
5. Language is clear, descriptive and complete 
a. | use specific words 31.3 62.6 5.2 0.8 
b. | choose the best possible words to describe my speech 38.7 54.1 6.4 0.8 
c. lt is important for me to be understood 67.7 27.9 2.9 1.5 
d. When needed, | clearly express my ideas in detail 40.8 51.0 6.9 1.3 
6. The ability to deal with several sources at one time and to 
consider alternative points of view 
a. | am willing to change my mind when given good reasons 58.3 37.8 3.1 0.8 
b. | offer and accept more than one alternative to problems 42.4 50.5 5.8 1.3 
c. | am willing to accept there may be more than one answer to 63.0 32.7 3.1 1.2 
a problem 
d. | am willing to accept other people’s solutions to problems 58.0 37.3 3.0 1.8 
Continued 
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Table 3-9 (continued) 


Rating Scale 
Attributes No 
Often |Sometimes |Not Yet | Response 

7. Has the interest and ability to find problems to solve 

a. | am not afraid to ask questions 48.0 42.9 8.3 0.8 

b. My questions are serious ones that | really want answered 43.0 51.0 49 1.0 

c. | ask questions until | am pleased with the answer 33.2 49.8 15.8 1.2 

d. Some of my questions are much harder than others 34.2 52.9 11.8 1.0 
8. All aspects of work are reviewed and checked 

a. | reread instructions and make sure everything has been done 38.0 54.5 6.5 1.0 

b. | want ali my work to be my best 63.6 33.6 1.9 0.9 

c. | try to correct errors before | finish a task 44.5 48.6 5.7 1.2 
9. The ability to understand one experience and apply it to 

new situations 

a. In a new situation, | figure out what | already know 42.2 52.8 4.0 1.0 

b. | remember and use important previous experiences 45.6 48.1 4.9 1.3 

c. | use ideas from the past to help me solve new problems 49.4 44.2 5.3 11 

10. Creativity 

a. | do work and activities because | want to 49.2 44.1 5.7 1.0 

b. | am willing to try things even when | am unsure of the result 51.5 43.5 ° 4.0 1.0 

c. | want others to tell me how | am doing 48.6 40.9 9.2 1.3 

11. Feel good about our thinking ability 

a. | enjoy finding problems to solve : 23.8 55.6 19.5 1.1 

b. | believe | can solve most problems ; 33.6 57.4 8.9 1.1 

c. | am curious and interested in most things 62.1 33.3 3.6 1.0 
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Table 3-10 

Grade 6 Social Studies 
Teacher Survey on Performance-Based Assessment 
Percentage Distribution of Teacher Responses 
June 1993 


Descriptors For Rating Scale 
Often: Student consistently demonstrates this attribute 


Sometimes: Student usually demonstrates this attribute, but not with consistency 


Not Yet: Student rarely, if ever, demonstrates this attribute 


Rating Scale 
Attributes No 
Often | Sometimes| Not Yet | Response 
1. Persistence: 
Continuing to work when the solution to a problem is not easily found 
a. stays on task but may take a break 46.0 43.2 10.4 0.5 
b. seeks other sources of data 34.4 49.0 16.1 0.5 
c. works independently to complete tasks/projects 50.0 ' 37.2 12.3 0.5 
2. Decreasing impulsivity: 
tlle before acting 
a. listens and understands directions before starting 49.3 43.9 6.3 0.5 
b. listens to the responses of others; therefore reduces the number 
of unnecessary or repetitive questions 48.5 41.5 9.2 0.7 
c. asks questions to clarify the task or direction 43.8 43.9 11.8 0.5 
d. analyzes the problem and develops a plan 39.8 44.8 14.7 0.8 
e. thinks before answering 49.8 41.1 8.2 1.0 
3. Listening To Others: 
Listens with understanding and empathy 
a. paraphrases others’ responses 24.0 3.2 22.3 0.5 
b. uses body language to respond 33.9 53.7 11.6 0.8 
c. asks questions related to the topic 46.4 42.7 10.4 0.5 
d. responds by acceptable actions or words 49.8 42.7 6.9 0.6 
4. Metacognition: 
Awareness of our own thinking 
a. verbalizes an understanding of the problem 39.0 48.6 11.8 0.6 
b. describes what is known to solve the problem 40.6 47.2 11.6 0.6 
c. identifies what else needs to be known 35.8 47.9 15.5 0.9 
d. develops and carries out a plan of action 41.1 45.0 13.3 0.6 
e. recognizes when the plan may need modification 33.0 45.0 21.4 0.6 
f. reflects/evaluates strategies used to solve the problem 27.2 45.3 26.6 0.9 
5. Precision to Language and Thought: 
Language is clear, descriptive and complete 
a. uses specific words (nouns, etc.) 39.8 51.1 8.5 0.6 
b. descriptors are used to enhance speech 32.1 50.4 16.8 0.7 
c. strives for coherence and conciseness 37.5 45.2 16.6 0.8 
d. elaborates and clarifies ideas 31.7 47.4 20.0 0.9 
Continued 


Table 3-10 (continued) 


Attributes 


Rating Scale 


Often 


Sometimes 


10. 


11. 


. Flexibility In Thinking: 


The ability to deal with several sources at one time and to consider 

alternative points of view. 

a. is ng to change his/her mind when presented with reasons 
to do so 

b. offers/accepts more than one alternative to a problem 

c. is willing to accept that there may be more than one answer 

d. is able to compromise (give up ownership) 


Questioning And Problem Solving: 

Has the inclination and ability to find problems to solve 
a. student is not hesitant to ask questions 

b. questions result from critical observations 

c. questions openly, is willing to probe 

d. demonstrates different levels of questioning 


Checking For Accuracy And Precision: 

All aspects of work are reviewed and checked 

a. rereads instructions and makes sure nothing has been 
omitted : 

b. takes pride in the accomplishment 

c. strives for quality through editing and proofing 


Drawing On Past Knowledge and Applying It To 

New Situations: 

The ability to abstract meaning from one experience and apply it 

to new situations 

a. determines what is known about a situation 

b. takes meaning from relevant past experiences 

Cc. aporepcaiely applies strategies used in the past in a new 
situation 


Ingenuity, Originality And Insightfulness: 
Creativity - 
a. student is self-motivated 


_b. takes risks and tests limits 


c. seeks and accepts feedback 


Wonderment, Inquisitiveness, Curiosity And The 

Enjoyment Of Problem Solving: 

A sense of efficacy as a thinker 

a. student derives pleasure from being a problem seeker 

b. student has confidence in his/her ability to soive problems 

c. has a curiosity about and an affinity with the world and the 
environment 


50.1 
47.4 


' 48.8 


38.3 
46.5 
43.1 


46.1 
43.1 


43.4 


No 
Not Yet | Response 
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Contexts for Learning 


General Description 

In June 1993, a sample of Grade 6 
Social Studies teachers and 
students from across the province 
completed separate questionnaires 
designed to identify and examine 
relationships among various 
contexts for learning and their 
effect on student achievement. 


Teachers reported on the variety of 
instructional strategies, writing 
activities, assessment tools, 
classroom resources, and other 
activities they used in their social 
studies classes. The results are 
presented in Table 3-11 (teachers). 


Students gave their opinions on a 
variety of topics and on various 
learning activities related to social 
studies. The results are presented 
in Table 3-12 (students). 


Teachers indicate that they use a 
wide variety of social studies 
related activities in their Grade 6 
classes. These activities includes 
co-operative learning, 
brainstorming, note taking, and 
class discussion. 


Student responses to the 
questionnaire indicate that 
students have a marked tolerance 
and openness to people different 
from themselves. A large majority 
of students indicated that they are 
accepting of differing opinions and 
of the importance of being 
responsible citizens; 84% of the 
teachers surveyed indicated that 
most of their students show respect 
for other’s opinions or viewpoints. 


Students responses were highest in 
the area of cooperation. For 
example, to the statement “It is 


group projects”, 95.7% of the 
students “agreed.” In a related 
area, 92.0% of the teachers 
surveyed indicated that most of 
their students demonstrated 
courteous behaviour when working 
with others. 


Relationship between 
Learning Environment 
and the 
Achievement Test 

No relationship or pattern was 
established by students achieving 
or not achieving the acceptable 
standard and their responses to 
the student questionnaire. It is 
also interesting to note there was 
no discernable pattern between 
achievement and instructional 
strategies. 


important for me to do my share in 


rrr 
Table 3-11 

Grade 6 Social Studies 

Teacher Questionnaire—Contexts for Learning 
Percentage Distribution of Responses 


Ee ee 


June 1993 
Less than Several 
Once | ,. 
Statement once a a times a No 
Never week week week / Every day | Response 
1. | use the followinginstructional strategies 

with my Social Studies students. 
a.’ Full class discussion - - 16.0 52.0 32.0 1.0 
b. Reading from a textbook - 32.0 20.0 48.0 1.0 _ 
c. Worksheets 12.0 32.0 24.0 28.0 a 4.0 
d. Cooperative learning _ 20.0 20.0 52.0 8.0 _ 
e. Library research 4.0 68.0 16.0 12.0 - _ 
f. Viewing films or videos = 80.0 12.0 8.0 - - 
g. Oral presentations _ ' 64.0 16.0 20.0 a _ 
h. Project work _ 52.0 24.0 16.0 4.0 4.0 
i. Note taking -_ 36.0 36.0 28.0 = = 
j. Brainstorming = 20.0 28.0 44.0 8.0 - 
k. | Webbing/mind mapping — 68.0 12.0 20.0 = -_ 
I. Learning or listening centres 56.0 28.0 _ 4.0 4.0 8.0 
m. Independent work _ 8.0 48.0 36.0 8.0 _ 
n. Problem-solving _ 20.0 32.0 40.0 4.0 4.0 
0. Teacher-directed inquiry _ 16.0 32.0 44.0 4.0 4.0 
p. Student-directed inquiry —_ 40.0 24.0 24.0 4.0 8.0 
q. Peer discussion — 20.0 20.0 44.0 8.0 8.0 
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Table 3-11 (continued) 


Statement 


My class does the following kinds of writing in 
Social Studies. 


. Paragraphs 

. Reports 

Stories 

. Writing about a personal experience 

Copying notes from the board 

Notes from class/group discussion 

. Letters 

. Summarizing information from a variety of 
sources 

i. Interviews 


sa -oa0 on 


| integrate the Social Studies program with other 


curriculum areas. 


| encourage students to make decisions and 
consider the consequences. 


Statement 


| encourage students to take action on an issue. 


Less than 


week 


oncea . 


Every day | Response 
es 


Once | Several 
a times a 
week week 
40.0 24.0 — 
12.0 12.0 - 
4.0 — 4.0 
4.0 — 4.0 
28.0 8.0 — 
24.0 32.0 - 
8.0 _ — 
32.0 36.0 - 
4.0 _ - 
20.0 24.0 12.0 
16.0 32.0 36.0 


No 


No Response 


Most of my students demonstrate courteous -_ 8.0 

behaviour when working with others. ; 

Most of my students can accept the role of 76.0 16.0 8.0 

either leader or follower in a group. 

Most of my students show respect for someone 84.0 8.0 8.0 

else’s opinions or viewpoints. 

The following activities are part of my grade 6 

Social Studies program. 

a. Field trips 72.0 28.0 = 

b. Simulation, role-play, or cooperative 84.0 16.0 = 
games 

c. Listening to guest speakers 96 0 4.0 - 

d. Interviewing parents and/or members of the 64.0 36.0 = 
community : : 

e. Special celebrations 

f. Making maps, charts, timelines, ono See sie 

100.0 - ~ 

and/or graphs 

g. Writing a letter 72.0 28.0 = 

Continued 
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Table 3-11 (continued) 


No Response 


Statement Yes No 
10. | use the following types of assessments with 
my Social Studies students. 
a. Multiple choice tests 88.0 12.0 
b. Vocabulary match 84.0 16.0 
c. Short answer 96.0 4.0 
d. True-false 84.0 12.0 
e. Extended response 96.0 4.0 
f. Interpretations of charts, diagrams, 100.0 0.0 
pictures, graphs, and maps 
g. Creation of a diagram, picture, mural, 
collage, or model to show understanding 
h. Oral presentation 
Students’ Teachers’ 
Statement Use Use 
| 
11. The following items were readily available 
in my classroom for my students and/or for 
my use in planning and instructing. 
a. Nelson.Intermediate Atlas 24.0 12.0 
b. Dictionary 36.0 = 
c. Encyclopedias 16.0 16.0 
d. Globe 28.0 8.0 
e. Newspapers 28.0 4.0 
f. Magazines 24.0 16.0 
g- Travel Brochures 20.0 16.0 
h. Films, videos 32.0 16.0 
i. Maps 28.0 = 
j. Computer 28.0 4.0 
k. Topic A: Basic Resources 
i. Politics and You (student text) 56.0 - 
ii. Politics and You (teacher's guide) 16.0 56.0 
iii. The Structure of Government (kit) 20.0 44.0 
iv. The Winds of Change: Indian 20.0 20.0 
Government (student texts) 
v. Working for Canadians: A Study of 32.0 12.0 
Local, Provincial , and Federal 
Government (student texts) 
|. Topic B: Basic Resources 
i. |The Greeks: People of the Past 52.0 8.0 
series (student texts) 
m. Topic C: Basic Resources 
i. Life in Changing China (student 56.0 4.0 
texts) 
ii. Life in Changing China (teacher's 28.0 60.0 
manual) 
n. Social Studies Teacher Resource Manual, 20.0 68.0 
Grades 4-6 
0. Focus on Research: A Guide to Developing 8.0 36.0 
Students’ Research Skills 
p. Teaching Thinking: Enhancing Learning 4.0 20.0 


Both Neither 
52.0 12.0 
64.0 _ 
36.0 32.0 
56.0 8.0 
40.0 28.0 
28.0 32.0 
20.0 44.0 
24.0 28.0 
64.0 8.0 
36.0 32.0 
24.0 20.0 
— 28.0 
8.0 28.0 
16.0 44.0 
24.0 32.0 
32.0 8.0 
36.0 4.0 
8.0 4.0 
4.0 8.0 
4.0 52.0 
4.0 72.0 
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Table 3-12 

Grade 6 Social Studies 

Student Questionnaire—Contexts for Learning 
Percentage Distribution of Responses 


June 1993 


Statement 


16. 
17. 


Participation is an important part of Social Studies. | 


It is important for me to do my share in group 
projects. 


Social Studies has been interesting this year. 
Social Studies is a difficult subject. 


Social Studies is useful for solving problems and 
dealing with situations in everyday life 


Violence should not be used to solve disputes 
between countries. 


Immigrants to Canada offer points of view that make 
Canada a better place. 


Other people have a right to an opinien that is 
different from mine. 


A person who moves to Canada from another culture 
should give up his/her own culture. 


| think that it is important to know about other 
countries and what is going on in the world. 


If people work together, they can help solve some 
of the world’s problems. 


When people wish to change a law, they should 
follow the established democratic practices, 
no matter how long it takes. 


There is no point in trying to think of solutions to local 
problems because they are too complicated. 


The responsibility for students doing well in school 
lies mainly with the student him/herself. 


Citizens of Canada are responsible for seeing that 
Canada’s environment is kept clean. 


It is important that | become a responsible citizen. 


People should not have to look after each other. 
They should look after themselves. 


Disagree; Undecided Agree apse 
1.1 11.3 87.6 ce 
1.4 2.8 95.7 0.1 
12.3 34.4 53.3 _— 
34.3 39.5 26.0 0.1 
22.7 36.5 40.5 0.3 
4.5 10.7 84.7 _ 
9.2 41.2 49.2 0.4 
1.0 3.6 95.2 0.3 
82.5 12.1 5.1 0.3 
2.3 12.0 85.0 0.7 
1.7 8.5 89.8 - 
6.3 40.6 52.6 0.4 
73.6 20.1 6.1 0.3 
6.3 16.0 77.3 0.4 
3.6 9.9 86.2 0.3 
1.7 9.0 89.3 0.1 
48.3 30.9 20.1 0.7 
(Continued) 
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Table 3-12 (continued) 


No 
Response 


Statement Disagree | Undecided 


18. | think that it is important to know about _ 5.4 18.5 
Canada’s past and present. 


19. People in Canada can meet their needs in 1.5 11.8 
different ways. 


20. The way that people meet their needs in China is 15.7 33.3 
just as good as the way we meet our needs in 
Canada. 
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Section 4 


| 
Grade 9 
Science 


In 1993 a number of assessments 
were carried out in Grade 9 
Science. The achievement test was 
administered to students province- 
wide. Performance assessment 
tests were administered to a 
sample of Grade 9 Science 
students. As well, a sample of 
Grade 9 Science teachers and a 
sample of Grade 9 Science students 
participated in a study to examine 
the relationship among various 
contexts for learning and 
achievement. The results from all 
these assessments follow. 


Achievement Test 
General Description 


The Grade 9 Science Achievement 
Test was divided into two parts. 
Part A had 60 multiple-choice 
questions, each with four 
alternatives. Part B had 5 
numerical-response questions, 
which required students to 
determine or calculate an answer 
and then record it on the answer 
sheet. 


The statistics for the total test and 
for the various components are 
based on the results achieved by 
28 613 students: 25 635 wrote the 
regular form and 2 978 
participated in the achievement- 
over-time study, as shown in Table 
4-2. This section of the report 
provides answers to the following 
questions: 


How many Grade 9 students 
wrote each form of the test or 
were absent or exempted? 


¢ What percentage of students who 


wrote the Grade 9 Science test 
achieved the acceptable 
standard? 


© What percentage of students who 
wrote the Grade 9 Science test 
achieved the standard of 
excellence? 


e What did Grade 9 students know 
and what could they do in 
Science? 


© What parts of the Science 
curriculum caused Grade 9 
students difficulty? 


Summary of Results 


Results show that 77.6% of - 
students who wrote the test 
achieved the acceptable standard 
and 17.1% achieved the standard 


- of excellence on the total test. The 


number of students achieving the 
acceptable standard was lower 
than expected and the number 
achieving the standard of 
excellence was higher than 
expected. . 


Content of the Test 


The Grade 9 Science Achievement 
Test was designed to reflect the 
Grade 9 Science curriculum 
standards. The scope of the test 
was limited to student learner 


expectations that could be 
efficiently measured on a paper 
and pencil test. 


The test consisted of 65 questions 
in six topics: Diversity of Living 
Things, Fluids and Pressure, Heat 
Energy: Transfer and 
Conservation, Electromagnetic 
Systems, Chemical Properties and 
Changes, and Environmental 
Quality. The questions were 
designed to measure achievement 
in two domains: Concepts and 
Skills. 


51 


Test Blueprint 


Table 4-1 presents the blueprint 
used to develop the Grade 9 
Science Achievement Test. 
Classification of each question by 
component and reporting category 
is indicated in the table. 


Table 4-] 
Grade 9 Science 
Achievement Test Blueprint 


June 1993 


Distribution of Questions by 
Reporting Category Total Number of 
A tC t re ast Emph ae 
ssessment Componeni ng Shite est Emphasis 
Diversity of Living Things 1,5, 10, 11, 12, 14, 56 2, 3, 4, 6, 52 12 (18%) 
Fluids and Pressure 7, 8, 9, 34, 48, 49, 51, 58 32, 35, 36, NR2 12 (18%) 
Heat Energy: Transfer and 15, 16, 17 18, 45, 46, 50, NR3 8 (13%) 
Conservation 
Electromagnetic Systems 19, 39, 40, 59, 60, NR1* 41, 42, 43, 44, 47 11 (17%) 
Chemical Properties and Changes 20, 21, 23, 24, 33, 38, 57 22, 37, NR4, NR5 11 (17%) 


Environmental Quality 


Total Number of Questions and 
Test Emphasis 


13, 28, 29, 30, 53 


25, 26, 27,31, 54, 55 


36 (55%) 29 (45%) 


11 (17%) 


65 (100%) 


*NR—numerical-response question 
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Student Part: icip ation the Grade 9 Science Achievement Results for students in French 


In June 1993, principals reported a Test, who were absent, or who Immersion or Francophone 
total population of 34 362 students were exempted. In total, 87.8% of programs were reported separately 
in Grade 9. Table 4-2 presents the the students reported to be in _ to participating schools and 
number and percentage Grade 9 in June 1993, wrote the jurisdictions. 
distribution of students who wrote Grade 9 Science Achievement Test. 
iene OC CC oC CCC oo 
Table 4-2 
Grade 9 Science 
Student Participation 
June 1993 
Number Percentage 


Category 


of Students of Students 


Other Students Who Wrote: 45 


Francophone— Taught in French, Wrote in French 123 0.4 
Francophone—Taught in French, Wrote in English 0 0.0 
French Immersion—Taught in French, Wrote in French 1 403 41 
French Immersion—Taught in French, Wrote in English 1 0.0 
Students Absent ; 2 041 5.9 


Students Exempted from Writing 2 181 6.3 


Categories of Exemption: 


. Special Needs Students 1171 3.4 


1 

2. Subject Was Not Taught This Term 427 1.2 
3. English as a Second Language Students 298 0.9 
4. Language of Instruction Was Not English 1 0.0 
5. Other (as approved by the Superintendent) ; 284 0.8 


Total Principals’ Reported Population 
Test Day, June 1993: 34 362 100.00 


Grade 9 Enrollment: September 30, 1992 35 182 


“Provincial results are based on test scores achieved by students who were instructed in English and who wrote the English form of the 
achievement test, and those students who were instructed in a language other than English or French and who wrote the English form of 
the test. They are indicated by the shaded area of the table. 
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Results in Relation to 
Standards 


Achievement test results are 
reported in relation to assessment 
and achievement standards. The 
Provincial Assessment 
Standard is the lowest score on a 
test that a student must achieve 
for his/her performance to be 
judged “acceptable” and/or 
“excellent” in relation to curricular 
expectations. The Provincial 
Achievement Standard refers to 
the percentage of students writing 
the test who are expected to 
achieve the Provincial Assessment 
Standard. 


Figures 4-1 and 4-2 and Table 4-3 
show the percentage of students 
achieving provincial assessment 
standards in relation to the 
provincial achievement standards 
on the total test and on 
components of the test. These data 
are based on the results of the 

28 613 students in regular 
programs who wrote the test. 


The percentage of students 
achieving the acceptable standard 
was lower than expected and the 
percentage of students achieving 
the standard of excellence was 
higher than expected. 


When interpreting these findings, 
it is important to remember that 
they are based on the test scores of 
only a portion (83.3%) of the 
students reported to have been in 
Grade 9 in June 1993, i.e., those 
students in regular programs who 
wrote the achievement test. These 
figures tell us nothing about the 
levels of achievement of students 
in Francophone and French 
Immersion programs, of students 
who were absent on the day of 
writing, or of those students who 
were exempted from writing. 
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Figure 4-1 

Grade 9 Science 

Percentage of Students Achieving Acceptable Standard on the — 
Total Test and on Components of the Test 

June 1993 ' 


100 


80 


Concepts Skills Total Test 


* Achievement standard—85% of the students in the province who wrote the test are 
expected to achieve the acceptable standard 


Figure 4-2 

Grade 9 Science 

Percentage of Students Achieving Standard of Excellence on the 
Total Test and on Components of the Test 

June 1993 


Total Test 


Concepts 


* Achievement standard—15% of the students in the province who wrote the test are 
expected to achieve the standard of excellence 


Immersion and Francophone 
students were not set separately 
due to the small numbers involved. 
Thus, these students are not 
included in any discussion of the 
numbers of students achieving 
standards. 


Earlier studies on the language of 
testing show that the standards 
developed for the English version 
of the test cannot be directly 
applied to French Immersion 
students writing the translation; 
standards for the French 


Table 4-3 

Grade 9 Science 
Students Achieving Standards on the Total Test and on Components of the Test 
June 1993 


Reporting Category 
Standard of Excellence 


Provincial |: 
Maximum | Assessment |Ac 
Possible Standard* 


Score (Raw Score) 


Students Achieving Assessment 
Standard 


on Both Skills and Concepts 


Total Test 65 52 

Skills Only .29 23 

Concepts Only 36 29 

Acceptable Standard Brg then dug gin EE 
Total Test 65 33 BB | 24 82 22214 Po 77.62 
Skills Only 29 15 Ha BB a 24 321 214130 | 74.8 
Concepts Only 36 18 oe BB: 24 321 22 987 

Below Acceptable Standard N/A N/A N/A 4214 


* The Provincial Assessment Standard is a score determined by standard-setting procedures and is the lowest score a student must 
achieve for his/her performance to be judged “acceptable” and/or “excellent” in relation to curricular expectations. 


See Appendix A. 


““ The Provincial Achievement Standard refers to the percentage of students expected to achieve the Provincial Assessment Standard. 


It should be noted that the actual 
percentages of students achieving 
standards on the total test for 
Grade 9 Science (17.1% and 77.6%) 
are based on the 28 613 students in 
the regular programs who wrote 
the test. 


If percentages are based on the 
total June population reported by 
principals (not including the 
French Immersion and 
Francophone students who wrote 
the test), the percentages achieving 
standards would be: 

14.9% achieving the standard of 

excellence, 

67.7% achieving the acceptable 

standard. 


If the percentages are based on the 
September 30, 1992, Grade 9 
enrollment (33 655), not including 
the French Immersion and 
Francophone students who wrote 


the test, the percentages achieving 
standards on the total test would 
be: 
14.5% achieving the standard of 
excellence, 
66.0% achieving the acceptable 
standard. 


It is emphasized that the 
percentages based on enrollment 
present the lowest estimate of 
achievement. It is highly likely 
that some of the students who were 
absent, exempt, or not accounted 
for could have achieved standards. 


The number of students achieving 
the acceptable standard and the 
standard of excellence for each 
school was analyzed to determine 
whether achievement in schools 
was below, meeting, or exceeding 
provincial achievement standards. 
Schools classified as meeting 
provincial achievement standards 


were those for which the difference 
between the actual number of 
students and the expected number 
of students achieving standards 
was not statistically significant. 
Differences are only reported when 
there is a 5% or smaller probability 
that a difference of that size could 
occur by chance. 


As can be seen in Table 4-4, about 
one-third of schools were 
significantly below the provincial 
achievement standard. School 
administrators and teachers 
should critically examine science 
achievement test results to 
determine what they are going to 
do to improve the level of student 
achievement. Although this is true 
for all schools, it is particularly 
important for those schools 
performing below the provincial 
achievement standard. 
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Table 4-4 


Grade 9 Science 
Percentage Distribution of Schools* Meeting Achievement Standards on the Total Test 
(N = 568) 
June 1993 
Percentage Distribution of Schools 
Significantly Below Not Significantly Different Significantly Above 
Provincial Achievement From Provincial Provincial Achievement 

L Standard Standard Achievement Standard Standard 

Standard of Excellence 7.0 77.6 

Acceptable Standard 33.6 63.6 ; 2.8 


“Schools with fewer than five students are excluded, as the statistical significance of the difference between the number actually achieving the 
standard and the number expected to achieve the standard when calculated and reported is not educationally meaningful. 


Reporting Categories 
Table 4-5 shows the total marks 
possible and the provincial raw 
score results for the reporting 
categories of the Grade 9 Science 
Achievement Test. 


It is important to stress that the 
averages on the various reporting 


categories cannot be directly 
compared with one another. 
Rather, the results shown in Table 
4-5 can best be used in conjunction 
with parallel tables in the 
jurisdiction and school reports. 
Variations in patterns of students’ 
responses to questions can help to 
indicate strengths and weaknesses 


in local educational programs. 
Statistics presented in this table 
are based on results achieved by 
26 653 (students who wrote the 
regular form and those who wrote 
Form T). 


Table 4-5 
Grade 9 Science 
Raw Score Resulis by Reporting Category 
June 1993 
i Number 
Reporting Category of Questions Average Score Standard Deviation 
Topic 
Diversity of Living Things 12 7.1 2.3 
Fluids and Pressure 12 7.7 2.5 
Heat Energy: Transfer and Conservation 8 4.3 1.6 
Electromagnetic Systems 11 7.2 2.2 
Chemical Properties and Changes 11 7.5 2.3 
Environmental Quality 11 7.3 2.4 
Learning Domain 
Skills 29 18.2 5.1 
Concepts 36 22.9 6.0 
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Results for Individual response (key) for each question is responses to questions can help to 


Multip le-Choice and also identified. Table 4-7 showsthe indicate strengths and weaknesses 
Numerical-Re sponse distribution of responses for each in local educational programs. 

7 numerical-response question. The 
Questions results shown in these tables can Statistics presented in these tables 
Table 4-6 shows the percentage of best be used in conjunction with are based on results achieved by 
students who chose each alternative the parallel tables in the 26 653 students (those who wrote 
(A, B, C, and D) for each multiple- jurisdiction and school reports. the regular form and those who 
choice question. The correct Variations in patterns of students’ wrote Form T). 


Table 4-6 


Grade 9 Science 
Results for Individual Multiple-Choice Questions 
June 1993 
Distribution of Responses (%) Topic: Diversity of Living Things 
Item 
A B Cc D Domain Curriculum Standard 
1 29.2 12.6 2.9 §5.2* Concepts ‘| define the meaning of “species” 
5 18.9 41 3.2 73.7* Concepts know which type of plant structure best retains moisture 
10 7.5 25.4 24.3 42.7" Concepts know taxonomic levels used in scientific classification 
11 74.7* 7.6 15.8 2.0 Concepts know how natural selection protects an insect population from 
; changes in the environment 
12 47.8* 9.2 34.1 8.8 Concepts identify types of specialization within a group of insects 
14 7.9 19.7 15.5 56.9* Concepis identity problems with selective breeding 
56 4.2 48.1 31.1* 16.5 Concepts identify a benefit to society of an application of 
selective breeding 
83.1" 5.0 5.3 6.5 Skills infer the relatedness of relationships between organisms 
46.2* 8.6 6.1 38.9 Skills infer the relatedness of species on the basis of their 
| classification 
3.1 8.6 16.3 72.0* Skills Classify oak trees using a dichotomous key 
6 59.8* | 19.2 17.1 3.9 Skills infer the adaption of birds to a food source 
52 ° 5.0 | 5.5 68.3" 21.2 Skills predict the habitat that an organism has adapted to based on 
observed characteristics 
*correct answer Continued 
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Table 4-6 (continued) 


Distribution of Responses (%) 


Topic: Fiuids and Pressure 


Curriculum Standard 


Cc Domain 
7 73.6* | 21.8 3.4 1.2 Concepts know that increased density results in increased buoyancy 
8 5.8 73.3* 75 13.3 Concepts know how pressure affects fluids in a siphon 
9 0.9 1.4 79.0* 18.6 Concepts know that pressure decreases as surface area increases 
34 25.2 4.6 5.8 64.3* Concepts know the relationship between fluid viscosity and 
temperature change 
48 15.7 48.8* 8.6 26.8 Concepts know how valves control the movement of fluids in a pump 
49 10.1 21.8 62.7* §.2 Concepts know the effect of resistance on objects moving in fluids 
51 23.7 48.1* 10.2 17.9 Concepts understand the relationship between buoyant force and the 
force of gravity ; 
58 8.5 60.7* 22.0 8.7 Concepts know how valves control the movement of fiuids 
32 11.4 70.1* 4.0 14.3 Skills infer that the reduced volume of a gas is related to the spaces 
between particles 
35 29.3 60.6* 6.0. 4.0 Skills predict the effect of temperature changes on the viscosity of 
motor oil 
36 51.5* | 21.3 58 21.3 Skills apply knowledge of the relationship between pressure, force, 
and area in a hydraulic system 
Distribution of Responses (%) Topic: Heat Energy — Transfer and Conservation 
Item | ; 
A B Cc D Domain Curriculum Standard 
15 25.3* | 55.0 2.5 17.1 Concepts know the effect of structural designs on the amount of cooling 
in houses 
16 11.8 80.5* 58 1.9 Concepts know how a solar collector converts the sun's energy into heat 
17 28.7 41.6" 13.7 15.9 Concepts know that metals are poor insulators 
18 5.6 11.9 58.3* 24.2 Skills relate the effect of variables when conducting an experiment 
45 8.1 16.1 54.6* 21.2 Skills judge the effects of heat transfer between solids and liquids 
46 14.1 33.1* 27.1 25.7 Skills infer the relationship between rate of temperature change and 
the mass of a fluid 
50 83.2* 47 5.9 6.0 Skills apply the principle of specific heat capacity 


*correct answer 
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Continued 


Table 4-6 (continued) 


Distribution of Responses (%) Topic: Electromagnetic Systems 
ttem 
A Cc D Domain Curriculum Standard 
19 7.4 46 65.7* 22.2 Concepts know the purpose of a thermocouple 
39 79.2* 8.8 4.2 7.7 Concepts know that changing electric current direction affects the 
direction of a compass needle 
40 5.1 4.4 11.0 79.4* Concepts understand how the strength of an electromagnet affects the 
speed of an electric motor 
59 6.2 8.5 74.5* 10.7 Concepts know how a bimetal strip can interrupt an electric circuit 
60 | 12.0 12.8 16.7 58.4* Concepts know the function of a fuse in a circuit 
| 41 26.0 14.5 44,9* 14.4 Skills predict a hidden circuit connection from circuit test results 
42 60.6* | 22.5 13.3 3.6 Skilis interpret information in a graph to show the reiationship of 
resistance and electric current 
43 3.9 48 86.7* 46 Skills apply knowledge of how switches work in a circuit 
44 50.9* | 21.3 7.0 20.7 Skills predict the effect of changing a circuit that has electric loads 
linked in series and parallel 
47 8.3 78.5" 6.2 6.9 Skills predict the effect on temperature caused by a resistor in a 
circuit ; 
Distribution of Responses (%) Topic: Chemical Properties and Changes 
Item ; 
A B Cc Domain Curriculum Standard 
20 14.5 70.9* 7.3 7.3 Concepts | identify a chemical change in different experimental results 
24 §.2 70.7* 15.1 8.8 Concepts | understand the effect of surface area on reaction rates 
23 3.9 17.4 2.4 76.3" Concepts identify the presence of acids and bases in household products 
24 11.1 21.2 13.0 54.6* Concepts know that the physical property density can be used to classify 
different materials 
33 AA 14.8 3.3 77.8* Concepts know a method of preventing corrosion 
38 15.2 4.7 3.6 76.5* Concepts understand that chemical change in materials produces heat 
57 69.6" | 21.3 3.5 5.6 Concepts understand that heating a solution causes an increase 
in solubility 
22 75 73.3" 77 11.5 Skills organize data that will show the effect of acid on a material 
37 1.8 3.9 84.9" 9.4 Skills interpret information from a diagram and apply understanding 
of melting point 4 


“correct answer 


Continued 
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Table 4-6 (continued) 


Distribution of Responses (%) Topic: Environmental Quality 
Item | 
A B Cc D Domain Curriculum Standard 
aE 

13 64.0* | 14.4 11.9 9.9 Concepts know the negative impact of chemical use on the environment 

28 12.2 12.5 17.7 57.6* Concepts identify the cause of low levels of dissolved oxygen in 
river water ; 

29 8.8 7.2 73.7* 10.2 Concepts know a procedure that could be used to measure the effect of 
chemical material on fish living in a river 

30 2.5 3.2 86.0* 8.2 Concepts identify an example of recycling used tires 

53 9.9 4.6 11.6 73.8* Concepts identify an environmentally friendty action in response to an 
environmental problem 

25 14.3 17.6 61.2* 6.7 Skills interpret data and identify correct experimental procedure 

26 71.6* | 11.0 14.9 2.4 Skills calculate the average number of organisms per drop of water 

27 35.3% | 11.9 40.3 12.4 Skills interpret data by relating the average population of an 
organism to the amount of water analyzed 

31 78 14.6 72.1* 5.4 Skills infer the cause of increased phosphate and nitrate levels 
in a river 

54 9.0 67.1* 10.1 13.7 Skills ‘ analyze data to find an increase of toxin levels in higher levels 
of a food pyramid 

55 64.4 18.8 10.1 6.6 Skills predict the effect of thermo pollution on health of trout living 

: in a lake 


*correct answer 


Table 4-7 
Grade 9 Science 
Results for Individual Numerical-Response Questions 
June 1993 
Distribution of 
Responses (%) 
Correct Incorrect No 
Item | Response | Response | Response Topic Domain Curriculum Standard Correct Answer 
1 44.6 | 55.1 0.3 Electromagnetic Concepts describe effect of resistance 4231 
Systems on electron flow 
2 72.1 | 27.3 0.6 Fluids and Skills predict changes in buoyant 4213 
Pressure force that result from changes in 
fluid density 
3 57.9 ;} 41.4 0.7 Heat Energy Skills interpret data and compare 3142 
heat conduction rates of materials 
4| 37.6 | 60.8 1.6 Chemical Skills infer and identify changes in 1122 
Properties materials as chemical or physical 
5 54.8 | 39.3 5.9 Chemical Skills calculate the density based on 1.75 
Properties mass and displacement 
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Examiner’s Observations 
Generally, the Grade 9 teachers 
who reviewed and set standards for 
the test felt that it was a good 
reflection of the Grade 9 Science 
program. They felt that the theme 
layout provided an excellent 
method for organizing the 


questions and giving real-world 
context to each question. There 

was a good range of question types 
and difficulties that were 
representative of the science, 
technology, and society emphasis of 
the program. The use of numerical- 
response questions gave more 


Use the following information to answer question 11. 


function. 


in plant growth. 


Over the past 20 years, the farmer kept careful records of different insect populations 
on his farm. His findings and observations are as follows: 


° Many insect species exhibit several distinct adult forms within the same species 
(polymorphism). The farmer noted that each form has adapted for a particular 
* From 1976 to 1989, the farmer found that he needed to use a higher concentration 


of insecticide each year to control the grasshopper population. 


* In 1987, the farmer discovered aphids in his clover crop. Aphids are tiny insects 
that use the juices from plants for food. Frequently, aphids can cause a reduction 


11. The most probable reason that the farmer needed to use a higher concentration of 
insecticide each year from 1976 to 1989 is that the insects 


74.7 *A. that survived previous applications passed their resistance on to their offspring 
7.6 B. were not affected by the insecticide because it was absorbed into the soil, water, 


and air 


15.8 C. reproduced quickly enough to replace those killed by the insecticide 


2.0 D. increased in size and weight 


Use the following information to answer question 37. 


George plans to use a paste wax to protect the paint on his car. He designs an 
experiment to test for the melting point of the wax. 


X represents a sample of the same wax in each test tube. 


Water at 25°C 


Water at 60°C 


Water at 90°C 


37. The melting point of the wax is 


1.8 A. lower than 25°C 

3.9 B. between 25°C and 60°C 
84.9 *C. between 60°C and 90°C 

9.4 D. higher than 90°C 


information about what students 
can do in science. 


Sample questions and commentary 
are provided. The correct answer is 
marked with an asterisk, and the 
percentages of students choosing 
each alternative is shown. 


Acceptable Standard 


For question 11, students had to 
know that natural selection 
protects an insect population from 
changes in the environment. 
Almost four out of five students 
achieving the acceptable standard 
know this. 


For question 37, students were 
required to interpret information 
from a graph and apply 
understanding of melting point of a 
substance. Most students can do 
this. 

To successfully answer 
numerical-response question 2, 
students had to predict the effect of 
density on buoyant force in four 
sugar solutions. Nearly four out 
five students achieving the 
acceptable standard can do this. 


For question 18, students had to 
identify the controlled variables of 
an experiment. Students achieving 
the acceptable standard but not 
the standard of excellence had 
difficulty doing this. 
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Use the following information to answer numerical-response question 2. 


A candy maker tested four different sugar solutions by placing a hydrometer in each. 


! 2 3 4 


2. Arrange the four solutions in order from lowest dehsity to highest density. 
72.1 Answer: 4,2,1,3 


Use the following information to answer question 18. 


| 


Mr. and Mrs. Brown decide to use curtain material that will make the 
house more energy efficient. They experiment to determine the best 
solar energy-absorbing material. They note the following variables: 


|. time of day 
1]. type of material 
Ill. length of time in the sunlight 
IV. surface area exposed to sunlight 


18. The variables that should be kept constant in this experiment are 


5.6 A. | and Il 
11.9 B. Iland IV 
58.3 *C. |, Ill, and IV 
24.2 D. Il, Ill, and IV 


Use the following information to answer question 53. 


Jane reports to the group. She ‘tells them that acid occurs when products of 
combustion of fossil fuels are released into the atmosphere, combine with 
moisture, and fall to the earth as acid rain or snow. The high acidity damages 
both water environments and human-made structures. In an effort to combat 
the effects of acid rain, governments have begun dumiping large quantities of 
calcium carbonate (limestone) into lakes to reduce their acidity. 


53 


9 


. For an environmentalist, which would be the most environmentally friendly means of 
dealing with the problem of acid rain? 


.9 A. Reducing the acidity with a base 


4.6 B. Putting acid-tolerant fish into the lakes 


11 
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.6 C. Adding calcium carbonate (limestone) to the lakes 
.8 *D. Reducing the amount of sulphur oxides released into the atmosphere 
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Overall, results show that the 

students who achieved the 

acceptable standard but not the 

standard of excellence, have an 

ability to . 

° know the effects of increased 
surface area 

e know that natural selection 
protects populations (question 11) 

* understand the effects of reaction 
rates 

* interpret information from a 
diagram (question 37) 

¢ know that electric currents cause 
magnetic fields 

© predict temperature changes 
caused by a resistor in a circuit 

® apply the principle of specific heat 
capacity — 


¢ identify environmentally friendly 


actions 

° know that a bimetal strip can 
interrupt an electric circuit 

¢ predict the effect of changing 
density on buoyant force 
(numerical-response 2) 


Many of the these students, 

however, did not do as well as 

expected in 

® defining the meaning of species 

* identifying controlled variables in 
an experiment (question 18) 

¢ identifying causes of reduced 
oxygen levels in a river 

® knowing the purpose of a 
thermocouple 

® interpreting a graph that shows 
an inverse relationship 

® predicting the effects of changes 
to parallel and series circuits 

* interpreting heat conduction data 

° using higher level skills 

° applying skills in novel situations 


Use the following information to answer question 58. 


During open-heart surgery, doctors often use artificial valves. If a person has 
a heart with a defective valve, an artificial valve can be used to replace the 
defective valve. This is illustrated below. 
Aorta Ball-type artificial valve 
a — 
(D 


Avra Z Antificial Bail in open 
y valve position 


Tk Left 

af ventricle 

; Ball in closed 
position 


58. When the ball of the vaive is in the “open” position, blood is 


8.5 A. forced out of the left ventricle into the aorta 
60.7 *B. supplied by the left atrium to the left ventricle 
22.0 C. forced out of the left ventricle into the left atrium 

8.7 D. supplied by the aorta to the left ventricle 


Use the following information to answer numerical-response question 5. 


Acrime lab analyst used displacement to calculate the density of a piece of 
metal. The mass of the metal is 70.0g. When the metal was added to the 
beaker, the displaced water overflowed into the cylinder. 


———, 


5. Calculate the density of the metal in g/cm®. DO NOT ROUND YOUR ANSWER. 


54.8 Answer: 1.75 g/cm? 


Use the following information to answer question 41. 


Sandy finds an old rail switching circuit board that does not have a circuit 
diagram. When she tests this circuit board using a dry cell and bulb, she 
obtains the following results: 


Contacts 
Tested 


41. The bulb will also light for connection 


26.0 A. JM 
14.5 B. JL 
44.9 *C. IM 
14.4 D. KL 


Standard of Excellence 


The following commentary 
highlights the skills and 
knowledge of students who 
achieved the standard of excellence. 


For question 53, students had to 
identify an environmentally 
friendly action in response to an 
environmental problem. Nearly 
95% of students achieving the 
standard of excellence were 
successful on this question. 


For question 58, students needed 
to know how valves control the 
movement of fluids. About nine out 
of ten students achieving this 
standard know this. 


Numerical-response question 5 
required students to calculate the 
density of a metal based on 
measured mass and volume 
displacement. Nearly all of these 
students can do this. 


Question 41 required students to 
predict hidden circuit connections 
from circuit test data. Some 
students achieving the standard of 
excellence had difficulty doing this. 
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Students who achieved the 
standard of excellence 
demonstrated more success than 
did other students when answering 
questions that required applying 
science concepts in novel or new 
contexts, identifying variables in 
experiments, and interpreting 
complex information or data. 
Specifically, students who achieved 
the standard of excellence could 
* comprehend the meaning of 
scientific and technological terms 
° infer relatedness of species 
* classify organisms 
¢ identify variables in experiments 
¢ interpret information and data 
about the environment 
° identify the purpose of different 
structures of living things 


* identify problems with selective 
breeding 

° predict the effects of temperature 
changes on liquid viscosity 

¢ predict habitat of organisms 

* evaluate environmental actions 
(question 53) 

® analyze how valves control the 
movement of fluids (question 58) 

¢ know the purpose of an electrical 
device 

° interpret information about 
chemical and physical changes 

° interpret information about heat 
conduction 

* calculate density (numerical- 
response 5) 


However, these students did not do 

as well as expected in 

° interpreting information about 
heat conduction and radiation 

° selecting a physical property that 
would classify materials in a 
group 

© understanding the application of 
average population of organisms 

* interpreting circuit diagrams and 
charts (question 41) 

® identifying the manipulated 
variable on a graph 

° identifying the benefit of a 
medical development on society 

® understanding the inverse 
relationship between resistance 
and conductor diameter and/or 
temperature. 


Performance-Based 
Assessment 

General Description 

A total of 697 randomly selected 
students from a sample of schools 
throughout the province were 
involved in performance-based 
assessment. 


Performance-based assessment 
was developed to assess students’ 
higher order thinking skills in real- 
life problem-solving situations. 


The tasks assessed aspects of 
science that could not be measured 
adequately by paper-and-pencil 
tests in which only the answer is 
recorded and marked. 


Students were asked to solve six 
real-life problems and were given 
concrete materials and/or 
information to solve these 
problems. These activities are 
described in Table 4-8. 

A group of experienced Grade 9 


Science teachers met in July 1993 
and established scoring standards. 
All student responses were scored 
following the standards set, with 
about 20% of papers being rescored 
to ensure marker consistency. 
Markers used holistic scales to rate 
student responses for problem 
solving and communication. 


The results are shown in Table 4-9 
and figures 4-3 and 4-4. 


Table 4-8 
Grade 9 Science 
Performance-Based Assessment Activities 
Activity Name Topic Program Area Learner Expectation 
1 Seed Dispersal Diversity of Living Things Nature of Science Observe seeds 
Infer seed dispersal 
Predict dispersal patterns 
2 Sugar Solution Fluids and Pressure Science and Technology Construct and calibrate a hydrometer 
Use a hydrometer to measure liquid 
density 
3 Sleeping Bags Heat Energy Science and Technology Design and execute an experiment to test 
insulation effectiveness 
4 Robot Arm Electromagnetic Systems Science and Technology Design, test, troubleshoot, evaluate, and 
construct an electromagnet 
5 Bulk Food Store Chemical Properties Nature of Science Develop an experimental procedure to 
identify white powders 
6 Hazardous Waste Environmental Quality Science, Technology, and Analyze information 
Society Examine perspectives 
Identify altematives 
Consider consequences 
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Table 4-9 
Grade 9 Science 
Performance-Based Assessment Results 


Communication 


Problem 
Solving 


Activity 1 Activity 2 Activity 3 Activity 4 Activity 5 Activity 6 
Student Achievement (%) (%) (%) (%) -  (%) (%) 
Beyond Grade 9 15.5 27.7 28.0 13.7 23.0 2.2 
At Grade 9 44.0 16.6 48.5 24.9 43.2 27.6 


Beyond Grade 9 
At Grade 9 


EUR 


Figure 4-3 
Grade 9 Science 
Percentage of Students Achieving At or Beyond Grade 9 Expectation (PBA), by Activity 
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Figure 4-4 
Grade 9 Science 
Percentage of Students Achieving At or Beyond Grade 9 Expectation (PBA), by Number of Activities 
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Examiner’s Observations 


The results show students could 


* solve problems using scientific 
inquiry skills 


© make simple observations and 
inferences 


* use a variety of strategies when 
given more materials than 
needed to solve a problem 


* account for the manipulated 
variable and responding variable 
in an experiment 


* get the correct solution if they 
could construct a device to solve 
the problem 


© arrive at the correct solution if 
they understood the problem and 
implemented an appropriate 
strategy 


The results show that students 
found it difficult to 


* solve open-ended problems 
successfully 


* suggest more than one alternative 
or perspective on an issue 


* construct a device to solve a 
practical problem 


* look at an environmental issue 
from different perspectives 


¢ make detailed observations 


* account for controlled variables 
in an experiment 


® identify and communicate the 
relationships found in a set of 
data or observations 


* communicate in forms other 
than—or in addition to—writing 
in sentences and paragraphs 


There are gender similarities and 
differences in students’ ability to 
solve problems; girls were better 
than boys in collecting and 
interpreting data and applying 
decision-making processes in an 
environmental issue. Boys were 
better at constructing devices to 
solve problems. Both were 


successful at applying basic 
inquiry skills in problem 
solving. 


* Overall, these results indicate that 


Grade 9 students have weak _ 
inquiry and problem-solving skills. 
They attained acceptable 
performances on only two of the six 
PBA activities. It is apparent that 
the new Grade 9 science program 
has not been fully implemented in 
Alberta schools and that many 
students are not receiving 
sufficient experiences in hands-on 
inquiry, hands-on problem solving, 


and environmental decision 
making. 


A more detailed description of the 
assessment tasks and scoring 
criteria, and samples of students’ 
performances are provided in the 
booklet Samples of Students’ 
Responses from the June 1993 
Grade 9 Science Performance- 
Based Assessment Tasks. 


Relationship between 
Performance-Based 
Assessment and the 
Achievement Test 


Generally, results show a positive 
correlation between achievement 
test scores and scores on the 
performance-based assessment. 
The highest correlations occur 
between the total achievement test 
score and each of the following 
performance-based activities: Seed 
Dispersal (0.50), Sugar Solutions 
(0.43), and Robot Arm (0.42). Also, 
positive but lower correlations 
occur for the other three activities: 
Sleeping Bags (0.27), Bulk Food 


Store (0.29), and Hazardous Waste 
(0.20). Students had greater 
difficulty demonstrating problem- 
solving skills in the performance- 
based assessment tasks than on 
the machine-scored component. 


Also, other possible relationships 
between student results on the 
Grade 9 Performance-based 
Assessment and student - 
performance on the Grade 9 
Science Achievement Test were 
explored by classifying students 
according to their performance on 
the achievement test. Students 
were split into three groups: 
students achieving the standard of 


excellence, students achieving the 
acceptable standard but not the 
standard of excellence, and students 
not yet achieving the acceptable 
standard. Table 4-10 contains the 
results of this analysis. 


Students achieving the standard of 
excellence on the achievement test 
also performed higher than other 
students on all performance 
assessment tasks. However, only 
2.5% of these students attained a 
level of excellence for Activity 6— 
Hazardous Waste, the most difficult 
task. 
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Table 4-10 
Grade 9 Science 

Relationship Between Achievement Test Results and 
Performance-Based Assessment 


Achievement Test Results (Percentage) 

Problem-Solving Component ~:~ Communications Component. - ae 
Standard of Below Acceptable OEE eae 

Excellence Standard 


Performance-Based _ Standard of 


Assessment Score Acceptable Standard 


___ Excellence” 


Acceptable ‘Standard 


ono 
eS 
N 
un 


Activity Six Activity Five Activity Four Activity Three Activity Two Activity One 


on--o 
tf) 
> 
oo 


Overall, students achieving the 
acceptable standard and not the 
standard of excellence on the 
achievement test found the 
performance-based tasks 
significantly more challenging than 
students who achieved the standard 
of excellence. They were able to 
respond best to the Sleeping Bags 
activity; however, only 28.3% 
achieved the acceptable standard on 
the Hazardous Waste activity. This 
highlights the difficulty students 
have addressing environmental 
issues from different perspectives. 


Students achieving below the 
acceptable standard had a low 
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success rate on five of the six 
activities. They had the greatest 
success on Activity 3—Sleeping 
Bags, where 60.2% and 54.5% of 
students achieved at or beyond 
Grade 9 expectations on problem- 
solving and communication skills, 
respectively. They had the least 
success with the Hazardous Waste 
activity, where 10.2% of these 
students achieved the acceptable 


standard for problem-solving skills. 


Contexts for Learning 


General Description 
In June 1993, 154 Grade 9 Science 
teachers and 532 students from 


across the province participated in 
a study designed to examine 
relationships among various 
contexts for learning and student 
achievement. Students responded 
to questions related to the 
attitudes outlined in the Program 
of Studies for Science. Teachers 
reported on the types and 
frequency of use of instructional 
strategies, activities, classroom 
resources, and manipulatives. 
Results of this study are reported 
in Table 4-11 and Table 4-12. 


Table 4-11 
Grade 9 Science 


Contexts for Learning: Percentage Distribution of Student Responses 


No 
Never Comment 


Statement Sometimes Rarely 
1. There are many ways to find an 2.3 22.0 2.0 <1.0 <1.0 
answer to a practical problem in ; 
science. 
2. | am confident that when | am 7.0 38.0 8.0 1.0 1.0 
presented with a problem in 
science, | can usually find a way to 
solve it. 
3. It is important to know some 36.0 18.0 2.0 <1.0 <1.0 
science in order to get a good job. 
4. | am good at science 8.0 40.0 8.0 3.0 3.0 
5. Science helps me: 
a. understand how things work 34.0 17.0 2.0 <1.0 <1.0 
b. prepare for high school 38.0 19.0 5.0 <1.0 3.0 
c. solve practical problems 24.0 22.0 5.0 2.0 1.0 
d. make decisions(env. qual) 27.0 26.0 6.0 2.0 1.0 
6. Science is useful for solving every 15.0 34.0 12.0 1.0 1.0 
day problems. 

_ 7. lenjoy doing science experiments 31.0 26.0 8.0 2.0 1.0 
8. | like to be challenged in science 20.0 31.0 13.0 7.0 3.0 
9. | feel successful in science 7.0 42.0 12.0 3.0 3.0 
10. The more | learn in science, the 24.0 25.0 7.0 3.0 2.0 

more interesting it becomes. 
11. I like to be presented with a 12.0 35.0 17.0 9.0 2.0 
problem in science, and then 
develop my own procedure to 
solve it. 
12. | would rather watch the teacher do 6.0 23.0 29.0 31.0 1.0 
a lab demonstration than do the 
investigation myself. 
13. Theories in science cannot be 8.0 31.0 19.0 14.0 14.0 
questioned or changed. 
14. If, after solving a challenging 
science problem, | found that other 
classmates had different results, | 
would: 
a. assume | made a mistake and 5.0 44.0 23.0 7.0 2.0 
that my results are incorrect 
b. assume the others made a 2.0 42.0 30.0 11.0 3.0 
mistake 
c. want to share my results with 22.0 24.0 8.0 4.0 3.0 
others 
15. | like to work in groups in science 54.0 13.0 2.0 2.0 1.0 
class. 
Continued 
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Table 4-11 (continued) 


technological or scientific 
equipment, | study or try to find 
out the correct safety procedures 
for its use. 


Statement Always Often Sometimes 
16. | enjoy doing science projects 24.0 35.0 27.0 9.0 
17. | have used the following sources 
of information this year to gain 
information for science research 
reports or science projects: 
a. experts 7.0 13.0 26.0 23.0 
b. references 20.0 31.0 28.0 11.0 
c. community resources 4.0 13.0 25.0 28.0 
18. | do science experiments at home. 2.0 3.0 22.0 33.0 
19. If a conclusion is going to have 15.0 23.0 34.0 8.0 
merit, then the experiment can. be 
repeated and the same results will 
be generated. 
20. | am careful to record exactly what 30.0 42.0 21.0 3.0 
| observe so that my conclusions 
are based on the most accurate 
data. 
21. Technological problems are solved 8.0 20.0 39.0 19.0 
by using the same methods or 
procedures each time. 
22. | find it easy to work with 20.0 32.0 31.0 10.0 
technological equipment 
23. Environmental problems can be 15.0 47.0 28.0 3.0 
solved with technological or 
scientific knowledge 
24. If 1 have a viewpoint on an 14.0 23.0 28.0 22.0 
environmental issue, | will try to 
convince a person who has an 
opposite viewpoint that they are 
wrong. 
25. When | am using a piece of new 26.0 33.0 27.0 5.0 


5.0 


3.0 


2.0 


9.0 


4.0 
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1.0 


8.0 
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Table 4-12 
Grade 9 Science 
Contexts for Learning: Percentage Distribution of Teacher Responses 


Statement Never Rarely | Sometimes Often iene N/A 
——————— ee 
1. | used the following instructional 
strategies with my Science 9 students 
this year: 
a Full class discussion 0.0 18 32.7 538 11.7 0.0 
b. Reading from a textbook 12 43 424 39.5 76 0.0 
c. Worksheets 0.0 20.6 476 288 29 0.0 
d. Smail Group discussion 17 22.1 49.4 24.4 23 0.0 
e. Research 12 25.7 52.9 13.5 18 0.0 
f. Oral presentations 13.5 339 43.9 76 16 00 
g. Project work 18 188 54.1 22.9 24 0.0 
h. Viewing films/video 47 28.5 55.2 11.0 0.6 0.0 
i. Lecture and note taking 12 13.4 43.6 36.6 52 00 
j. Lab investigations 0.0 17 233 5B4 18.6 0.0 
k. Demonstrations 0.0 §2 40.1 448 99 0.0 
|. Field Trips 30.5 442 11.6 17 12 17 
2 How often did your grade 9 Science 
class do the following kinds of writing 
this year? 
a Joumal entries 69.8 13.6 83 59 18 06 
b, Essays 233 465 24.4 47 0.6 0.6 
c. Summaries 29 20.6 429 29.4 3.5 0.6 
a. Descriptions 24 113 49.4 31.5 42 12 
e. Note taking 06 87 372 43.6 9.9 0.0 
f. Ideas from research 53 23.7 50.9 178 24 00 
g. Ideas from group/class discussion 1.7 16.3 488 28.5 47 
h. Lab write-ups 0.0 29 20.9 52.3 23.8 0.0 
i. Reports 35 28.2 471 17.1 3.5 0.6 
j. Generating questions 8.1 227 39.5 22.7 64 06 
k. Inquiry activity write-ups 42 13.7 36.3 B3 10.7 18 
3. This year, my students have 
presented projects in the following 
formats: 
a Written 23 7.0 433 38.0 9.4 0.0 
b. Story 58.7 23.4 126 18 06 3.0 
c. Essay 29.5 28.3 31.9 72 18 12 
d Display 13.1 252 423 13.7 18 0.0 
e. Videotape 770 10.9 73 18 06 24 
f. Newspaper article 61.1 20.4 12.0 3.0 12 24 
g. Poster series 25.4 20.1 42.0 10.7 18 0.0 
Continued 
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Table 4-12 (continued) 
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Statement Never Rarely Sometimes Often Always NWA 
h. Audiotape 849 84 18 18 0.0 3.0 
i. Scale model 443 21.0 263 42 12 3.0 
j. Play 747 127 84 06 06 3.0 
k. Slides and Commentary 83.6 91 48 0.0 0.0 24 
I. Working model 22.8 216 49 10.8 12 18 
4. lencourage a variety of formats for 47 129 42.1 240 11.7 47 
presenting research reports 
5. [encourage students to design their 23 15.7 483 25.6 8.1 0.0 
own experiments 
6. | demonstrated that practical 0.0 18 25.1 §3.2 19.9 0.0 
problems may have a variety of 
viable solutions 
7. How often did your students have a 23 22.4 55.2 14.0 58 06 
choice about how they would design 
their investigations? 
8. My students participate in at least one 0.0 53 28.1 25.1 415 0.0 
"hands-on" activity in science each 
week 
9. My students do at least one lab activity 0.0 58 22.2 292 42.7 0.0 
in science each week. 
10. Through modelling and discussion, 0.0 3.5 26.3 39.2 31.0 0.0 
my students are encouraged to . 
realize that a particular point of view 
must be supported with factual data. 
11. When my students were presented 
with an open-ended problem where 
they were expected to develop their 
own plan, most of them appeared: 
a Motivated 12 28 39.2 122 8.4 12 
b. Bored 82 42.1 413 75 - 19 
c. Frustrated 19 179 61.5 179 - 0.6 
d. Confident 12 16.8 46.0 342 12 06 
e. Insecure 89 31.6 443 152 - - 
Continued 


Table 4-12 (continued) 


12. 


13. 


Statement Yes No N/A 
= 
Which of the following types of 
assessment did you use with your 
science students this year? 
a Multiple choice tests 98.8 12 _ 
b. Short answer 99.4 06 - 
c. True-false 84.9 13.4 1.7 
d. Extended response 94.8 29 23 
e. Performance assessment 70.3 25.0 47 
f. Lab write up 92.7 23 _ 
g. Presentations 715 279 06 
h. Checklists 372 59.9 29 
i. Anecdotal records 35.5 62.2 23 
j. Student self-evaluation 424 | 57.0 0.6 
k. Vocabulary match 79.1 19.8 12 
|. Interpreting diagrams etc. 99.4 0.0 06 
m. Creating diagrams, etc. 80.5 16.5 3.0 
and/or modeling to show 
understanding 

n. Problem solving 97.7 17 06 
Were the following activities a part 
of the grade 9 science program this 
year? 
a Field trips 36.0 628 12 
b. Science fair 2.7 715 17 
c. Science Olympics 19.8 785 17 
d. Listening to quest speakers 37.2 61.6 12 
e. Decision making 93.0 58 12 
f. Writing assignments 95.9 3.5 06 
g. Completing handouts 94.2 58 = 
h. Watching demonstrations - 98.8 12 _ 
i. Lab activities/experiments 100.0 - _ 
j. Research projects 87.8 11.6 06 
k. Making models 72.1 26.7 12 
|. Participating in debates 488 488 23 
m. Problem solving/small group 91.3 64 23 
n. Problem solving w/ STS 88.4 10.5 12 
0. Open-ended investigations 843 14.0 1.7 
p. Analyzing photo/graphfable 98.3 17 - 


Continued 
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Table 4-12 (continued) 
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Statement Yes ’ No NWA 
q. Using a computer 22.1 75.6 23 
r. Writing frequent quizzes 721 273 0.6 
s. Working in small groups » 953 35 12 
t Writing in a journal 17.4 802 . 23 
u. Completing exercises 97.7 17 06 
v. Generating solutions 95.9 23 1.7 
w. Evaluating designs 80.8 19.2 - 
x. Taking action toward an issue 49.4 49.4 23 
y. Evaluating effects of action 59.3 38.4 23 

14. This year, the following items were 

readily available in the science 
room for my students or my use in 
planning and instructing: ; 
a Science Directions 9 TRM 97.1 17 12 
b. Science Directions 9 ST 97.1 23 “06 
c. Sci+ Tech. & Soc. TRM 453 52.3 23 
d. Sc. + Tech. & Soc. ST 366 422 12 
e. Water Lit. Prog. —AB. Env. 33.1 663 0.6 
f. Space Age Agr. Res. 11.0 86.6 23 
g. Energ. Lit. Series(SEEDS) 32.6 645 29 
h. Aquatic Invert. Mon. Prog. 16.3 81.4 23 
i. Tables or lab counters 91.9 76 0.6 
j. Sinks for lab stations 878 11.0 12 
k. Films/videos 85.5 13.4 12 
|. Equipped 88.4 8.1 35 
m. Basic Mat. for Topic 1 93.0 41 29 
n. Basic Mat. for Topic 2 92.4 41 3.5 
o. Basic Mat. for Topic 3 93.6 3.5 29 
p. Basic Mat. for Topic 4 88.4 87 29 
q. Basic Mat. for Topic 5 93.6 29 35 
r. Basic Mat. for Topic 6 802 163 35 


Student Attitude 


Questionnaire 
The student questionnaires reveal 
that students generally feel that: 


° it is important to know some 
science in order to prepare for 
high school and to get a good job 


* science helps them understand 
how things work and to solve 
problems 


* it is important to record exactly 
what they observe so that their 
conclusions are based on the 
most accurate data 


Relationships between Student 
Attitudes and the Results on 
the Achievement Test 

Students achieving the standard of 
excellence on the achievement test 
understand that science helps 
them solve problems. They 
indicated that science will help 
them get ready for high school and 
to get a job. This group of students 
reported greater curiosity about 
events and objects in the natural 
world. They were more likely to be 
open-minded and thought that 
critical thinking was important. 

As a group, they also enjoyed doing 
experiments. 


Of the students achieving the 
acceptable standard, but not the 
standard of excellence, only 36.6% 
indicated they were often good at 
science. Also, only 56.1% indicated 
they often enjoyed doing 
experiments and 55% often enjoyed 
doing projects. However, this group 
of students did report a tendency to 
respect accuracy and precision, be 
critically minded and appreciate 
the contribution of science for 
solving problems. Most of these 
students liked to work in groups. 
Many of the students achieving 


below the acceptable standard 
indicated that learning science 
would help them get a job and that 
honesty in reporting is important. 
Only 23.3% of these students 
thought they were good at science. 
Also, only 58.7% of students in this 
group indicated they often enjoyed 
doing experiments in science. As a 
group, they showed a lower 
tendency to be critically minded. 


Teacher Questionnaire 
The teacher questionnaire reveals 
that teachers reported 


using a variety of instructional 
strategies with their Science 9 
students, with full class 
discussions and lab 
investigations the most 
frequently mentioned strategy. 


having students do various © 
writing, with note taking, lab 
write-ups, and inquiry activity 
write-ups the most frequently 
mentioned activities. 


having students do at least one 
investigation per week. 


* demonstrating to students that 
practical problems can have 
more than one solution. 


© using a variety of assessment 
methods. 


Journal writing was not being 
emphasized in schools—69.8% of 
teachers never had students write 
in a journal. This is of interest, 


because journal writing can be 


used for student self-assessment. 


Students tended to present their 
projects in written form (90.7%); 
however, they rarely write stories 
(only 15%) or essays (only 40.9%). 
In the future, there may be an 


opportunity for teachers to explore 
language communication in 
science. 


Teachers used a variety of 
assessment types, which tend to be 
teacher directed. Student self- 
evaluation was reported to be used 
by only 42.4% of the teachers. 


Enrichment activities were not 
emphasized—for example, only 
36.0% of teachers reported having 
their classes go on field trips, and 
only 37.2% of teachers reported 
having students listen to guest 
speakers. Enrichment activities 
can provide opportunities for 
students to see Science, 
Technology, and Society 
connections, 


Only 48.8% of teachers reported 
having their students participate 
in debates. Debates can provide an 
excellent opportunity for students 
to appreciate the different sides to 
a STS issue. Also, only 49.4% of 
teachers reported having students 
take action toward an STS issue. 


Although there is a technology 
focus in the Grade 9 science 
program, only 22.1% of the 
teachers reported using computers 
for instruction. 


Relationships between Context 
for Learning and the Results 
on the Achievement Test 
Possible relationships between 
reported teacher behaviours and © 
student performance on the 1993 
Grade 9 Science Achievement Test 
were explored by classifying 
students according to their 
performance and looking for 
differences in the frequency of 
responses on the teacher survey. 
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Students were split into three 
groups: students achieving the 
standard of excellence, students 
achieving the acceptable standard 
but not the standard of excellence, 
and students not yet achieving the 
acceptable standard. Student 
achievement results for 532 
students were matched with 
survey responses of their teachers 
(154 teachers). 


Teaching behaviours as reported 
by teachers of students achieving 
the standard of excellence were 
compared to the responses of 
teachers of students who had 
achieved the acceptable standard 
and to the responses of teachers of 
students who had not yet achieved 
the acceptable standard. 


For students achieving the 
standard of excellence, 93% of their 
teachers reported emphasizing 
technological problem solving and 
societal decision making, whereas 
for students achieving below the 
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acceptable standard only 73.9% of 
their teachers indicated 
emphasizing these skills in their 
class. Also, for students achieving 
the standard of excellence 76.1% of 
their teachers indicated that they 
encouraged students to support 
their points of view with factual 
data compared to 43.4% for 
teachers of students who had not 
achieved the acceptable standard. 


For students achieving the 
standard of excellence, their 
teachers reported having students 
write about generating questions 
(43.7%), write about generating 
ideas from discussion (50.7%), and 
write up inquiry activities (81.7%) 
more frequently than the teachers 
of students who had not yet 
achieved the acceptable standard 
(8.6%, 13.0%, and 47.8% 
respectively). 


For students who had achieved the 
standard of excellence, their 
teachers indicated they placed a 


greater emphasis on problem 
solving (93.0%) than the teachers 
of students who had not achieved 
the acceptable standard (73.9%). 
Also, for students who had 
achieved the standard of excellence 
52.1% of their teachers indicated a 
preference for having students do 
self-assessment compared to 39.1% 
of the teachers of students who had 
not achieved the acceptable 
standard. 


Students who had achieved the 
standard of excellence had teachers 
who indicated a greater preference 
to supplement their programs with 
additional instructional materials 
than teachers of students who had 
not achieved the acceptable 
standard. Also, students who had 
achieved the standard of excellence 
had teachers who indicated a 
greater usage of computers (28.2%) 
than the teachers of students who 
had not achieved the acceptable 
standard (4.3%). 
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Section 5 
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Achievement by Gender 
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Information on the gender of 
students who wrote the provincial 
achievement tests has been 
collected, analyzed, and reported 
since 1989. 


This section of the report answers 
the following questions: 


¢ What is the proportion of males 
and females who wrote the 1993 
achievement tests? 


¢ Is the percentage of males and 
females meeting standards the 
same in each subject? 


¢ Are the 1993 results for 
individual subjects similar to or 
different from those of 1989 
through 1992? 


Observations and 
Discussion 


What is the proportion of 
males and females who wrote 
the 1993 achievement tests? 


Results for 1993, presented in 
Figure 5-1, reveal that more males 
than females wrote the 
achievement tests at each grade 
level. This pattern is similar to the 
data from 1989 through 1992, with 
the exception of 1990 when more 
females than males wrote the 
Grade 9 English Language Arts 
test. 


Figure 5-1 
Number of Achievement Tests Written by Gender 
June 1993 
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Is the percentage of males and 
females meeting standards the 
same in each subject? 


Results for 1993, presented in 
Figures 5-2 and 5-3, reveal that 
more females than males achieved 
the acceptable standard and the 
standard of excellence in Grade 
Level 3 Language Learning and 
Grade 6 Social Studies. In Grade 9 
Science, however, males 
outperformed females, with higher 
percentages achieving both the 
acceptable standard and the 
standard of excellence. 


Are the 1993 results for 
individual subjects similar to 
or different from those of 1989 
through 1992? 


The 1993 pattern of results for 
Grade Level 3 Language Learning 
is similar to the Grade 3 results 
from 1989. A similar pattern was 
found in both the 1990 Grade 9 
and 1992 Grade 6 English 
Language Arts results. 


The pattern for the 1993 Grade 6 
Social Studies results is similar to 
the 1989 Grade 6 and the 1992 
Grade 3 Social Studies results: 
females outperformed males at 
both the acceptable standard and 
the standard of excellence. In 1991, 
however, males marginally 
outperformed fernales in Grade 9 
Social Studies. 


The 1993 Grade 9 Science results 
show a similar pattern to the 1989 
Grade 9 and the 1990 Grade 6 
Science results: males 
outperformed females, particularly 
at the standard of excellence. In 
1991, however, males and females 
showed similar levels of 
achievement in Grade 3 Science. 


There are gender differences in the 
levels of achievement attained by 
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Figure 5-2 

Number and Percentage of Students Achieving Acceptable 
Standard on the Total Test by Gender 

June 1993 
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Figure 5-3 

Number and Percentage of Students Achieving Standard gd 
Excellence on the Total Test by Gender 

June 1993 
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*15% of students were expected to achieve the standard of excellence on the total test. 
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explore gender differences within 
their own settings. 


males and females on the 
provincial achievement tests in the 
three grades and subjects tested in 
1993. Since individual jurisdiction 
and school results may show 
patterns that differ from the 
province-wide results, school 
personnel are encouraged to 


We welcome any comments 
regarding observations or thoughts 
on gender differences in 
achievement. 


Section 6 
{ 


Achievement by Age 


Information on the age of * What is the age distribution of Age Groups 
students who wrote the provincial students who wrote the 1993 Wika dellavemant teeta were 
achievement tests has been achievement tests? ; ; ; 
written in June 1993, students 

collected, analyzed, and reported ages in both years and months 
since 1990. ¢ What relationship, if any, does wore écltected. Based on this 

: : age have withectiovement eee information, each student was 
This section of the report answers measured by the 1993 provincial categorized as being in the first or 
the following questions: achievement tests? & 


the last half of his or her age year 


Table 6-1 
Number of Achievement Tests Written.by Age* 
June 1993 


Age Grade Level 3 
(6 month intervals) Language Learning Grade 6 Social Studies Grade 9 Science 


Number Number 


as 
7 (84-89 mos.) | 
7.5 (90-95 mos.) aa pee 
Pera al 


8 (96-101 mos.) 


et: Dood 


10(120-125 mos.) | 


10.5 (126-131 mos.) 


S 


| 00 
13 (156-161 mos.) 
13.5 (162-167 mos.) 
14 (168-173 m 


a 


[“Woroler 


*Age on the test day (June 8, 1993) 
An empty area indicates that there were no students 
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(e.g., 7 years = 84 to 89 months, 
7.5 years = 90 to 95 months). Table 
6-1 indicates the number and 
percentage of achievement tests 
written by age group and shows 
the exact range, in months, used to 
determine these data. Figure 6-1 
and Figure 6-2 show the proportion 
of students in each age group who 
achieved standards in each grade. 
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Figure 6-1 
Percentage of Students Achieving Acceptable Standard on the 
Total Test by Age 
June 1993 
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Figure 6-2 

Percentage of Students Achieving Standard of Excellence on the 
Total Test by Age 

June 1993 
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Observations 
and Discussion 


What is the age distribution of 
students who wrote the 1993 
achievement tests? 


Of the Grade Level 3 students 
tested in 1993, 42.5% were 8 years 
olds and 51.4% were 9 years old at 
the time of testing. Almost 5% of 
the students tested were 10 years 
old or more. Thirty-five of these 
students were aged 12, 13 or 14. 
Sixty-eight students were reported 
to be 7 years old or younger. 


In 1993, 40.7% of tested Grade 6 
students were 11 years old at the 
time of testing and 51.8% were 12 
years old. Of the tested students, 
fewer than 1% were younger than 
11 years old and 6.5% were older 
than 12 at the time of writing. 

’ Eighty students who wrote the 
Grade 6 test were 14 or 15 years 
old on that day. 


For the year 1993, 42.3% of tested 
Grade 9 students were 14 years old 
on June 8 and 49.9% were 15 years 
old. Of the students tested, 0.3% 
were younger than 14 years old on 
the day of testing and 7% were 
older than 15. Ninety-four students 
were 17 years or older on the day 
of testing. 


While the number of students 
writing the Grade Level 3 test who 
were reported to be 7 years old and 
younger was relatively small, this 
figure may, nevertheless, be 
spuriously high due to reporting 
error. Caution, therefore, is 
advised when making inferences 
about participation rates and 
levels of achievement for students 
in this age range. 
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What relationship, if any, does 
age have with achievement as 
measured by the 1993 

provincial achievement tests? 


In Grade Level 3, proportionally 
fewer students who were older 
than 9.5 years achieved the 
acceptable standard or the 
standard of excellence. 


On the Grade 6 test, 
proportionately fewer students 
aged 12.5 years and older achieved 
the acceptable standard and the 
standard of excellence. 


For Grade 9, there is an overall 
negative relationship between age 
and student achievement. Younger 
students generally achieve higher 
than do older students with regard 
to both the acceptable standard 
and the standard of excellence: 


These data indicate that, in 
general, students who are older 
than the median age for their 
grade did not achieve as well as did 
younger students who wrote the 
test. There is a considerable drop 
in the proportion of students 
meeting standards among those 
whose age is at the upper range of 
the mainstream and beyond. 


‘| 
Section va 


i 
Achievement by 
Grade Level 


This section answers the following 
question: 


What percentage of grades 3, 

6, and 9 Alberta students 

¢ may be achieving beyond 
grade level 

* are achieving at grade level 

* are not yet achieving at grade 
level 

as measured by the June 1993 

achievement tests? 


Within each curriculum, specific 
outcomes, expressed as the 
knowledge, skills, and attitudes to 
be acquired and developed, are 
arranged into sequences reflecting 
the developmental nature of 
learning and expectations for a 
learner’s increasing competence 

in the subject area. 


The 1993 achievement tests were 
designed to measure student 
achievement within the given 
grade level. However, a number of 
questions on the achievement tests 
could be considered to measure 
more advanced levels of 
performance. For reporting 
purposes, we have classified 
students within the grade levels 
framework based on the following 
three definitions: 


1. May Be Achieving Beyond 
Grade Level 
Students who may be achieving 
beyond grade level are those 
students who achieved the 
standard of excellence for both 
major components of the test. 


2. Achieving At Grade Level 
Students who are achieving at 
grade level are those students 
who achieved the acceptable 
standard on the total test and 

_those students who achieved the 
standard of excellence on the 
total test but not on both 
components. 


3 


. 


Not Yet Achieving At Grade 
Level 

Students who are not yet 
achieving at grade level are 
those students who did not 
achieve the acceptable standard 
on the total test. 


This classification is a different 
way of viewing the results from 
the achievement test, which 
means that the numbers 
reported differ from those 
indicated for percentage of 
students achieving standards. 


Results from the 1993 
achievement tests showing 
distributions of grade level 
achievement are presented 

in Figure 7-1. This analysis 
shows the wide range of student 
achievement within a nominal 
grade level. 
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Distribution of Grade Level Achievement 
June 1993 
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Achievement Over Time 


An important task of Alberta 
Education is to measure and report 
changes in student achievement 
over time. Since 1983, the Student 
Evaluation Branch has been 
measuring the achievement of 
students in grades 3, 6, and 9 in 
the subject areas of language arts, 
mathematics, science, and social 
studies. The administration of the 
Achievement Testing Program 
follows a four-year cycle for each 
subject within each grade level. 

In 1985, 1989, and 1993, students 
in Grade Level 3 Language 
Learning, Grade 6 Social Studies 
and Grade 9 Science were tested. 


This section of the report answers 
the following question: 


* Has achievement, as measured 
by individual provincial 
achievement tests, changed since 
1985 and 1989? 


Two studies were carried out. The 
first was designed to compare 
achievement since 1985 on the 
multiple-choice questions in Grade 
Level 3 Language Learning, Grade 
6 Social Studies, and Grade 9 
Science. The second study was 


designed to compare achievement 
since 1985 and 1989 on the written- 
response part of the Grade Level 3 
Language Learning test and the 
Grade 6 Social Studies test. 


Descriptions of the designs and 
results of these special studies 
follow. 


Multiple-Choice Study 
Design and Methodology 


A random sample of students was 
selected, by school, in each of grades 
3, 6, and 9, to participate in the 
achievement-over-time study. In 
each grade, the achievement-over- 
time (AOT) sample was divided into 
three sub-groups. At each grade 
level, one sub-group, referred to as 
the AOT93 group, wrote the regular 
1993 achievement test. Students in 
the remaining sub-groups wrote one 
or the other of two AOT test forms 
that were specifically designed for 


' this study. These clusters of 


students are referred to as the 
AOTS89 group or the AOT85 group, 
depending on which AOT test form 
they wrote. 


Sampling of the schools involved in 
the study was carried out 
independently in grades 3, 6, and 
9. Only schools having 18 or more 
students in regular English 
programs in 1992, in the grade 
being sampled, were included in 
the groups from which samples 
were drawn. The sample sizes were 
targeted for approximately 3 000 
students at each grade level. Since 
the sampling was carried out 
independently at each grade level, 
some schools were randomly 
selected to participate at more 
than one grade. Students within 
each class were randomly assigned, 
by the Student Evaluation Branch, 
to write either the AOT93, the 
AOTS839, or the AOTS85 test form. 


In each grade, the 1989 and 1985 
AOT test forms included questions 
taken from the 1993 achievement 
test for that grade and portions of 
the original achievement test from 
1989 or 1985. Table 8-1 shows the 
composition of these, and the 1993 
AOT test form, for each grade. 
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Table 8-1 


Design of the Achievement-Over-Time Tests 
Number of Questions from Previous and Current Tests 


1993 
AOT Test 


Grade Level 3 Language Learning 
Number of 1985 Questions 
Number of 1989 Questions 
Number of 1993 Questions 


Grade 6 Social Studies 
Number of 1985 Questions 
Number of 1989 Questions 
Number of 1993 Questions* 


Grade 9 Science 

Number of 1985 Questions 
Number of 1989 Questions 
Number of 1993 Questions 


1989 1985 
AOT Test AOT Test 

0 21 
21 0 
19 19 

0 28 
30 0 
20 22 

0 23 
20 0 
45 42 


“Three questions on the 1993 Grade 6 Social Studies Achievement Test were dropped. These questions were dropped from the AOT93 form 
and did not appear on either AOT89 or AOT85. 


The 1989 items on the AOT89 form 
were used to compare 1993 student 
performance to student 
performance on these items in 
1989. The 1985 items on the AOT85 
form performed a parallel function. 
Before these comparisons were 
undertaken, however, it was 
necessary to verify that the 
students selected to participate in 
the special study were 
representative of students 
throughout the province. This was 
achieved by comparing AOT 
student performance on the 1993 
items with provincial performance 
on the same items. The 1993 items 
on the AOT forms also allowed for 
the calculation of a 1993 test score 
for each of the students in the AOT 
study. 


Experience has shown that schools 
participating in the AOT study, like 
most other schools, prepare their 
students to write achievement tests 
by administering earlier versions of 
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those tests as practice. Questions 
from these earlier versions are the 
same ones used in the AOT study. 
Thus, scores for AOT students 
could be spuriously high due to the 
effects of practice. To check this 


. possibility, a questionnaire was 
-sent to teachers of the AOT groups 


to elicit information about student 
preparation practices. Based on 
teacher responses to the practice 
effects questionnaire, AOT 
students were divided into 
“practice” and “no practice” groups 
depending on whether or not they 
were reported to have used either 
1989 or 1985 tests in preparation 
for writing the 1993 achievement 
test. Group performance on the 
1993 items was compared to 
determine if the “practice” and “no 
practice” groups were of equal 
ability. Their performance on the 
1989 (or 1985) items was then 
compared to determine if there 
were differences that could be 
attributed to the effects of practice. 


Following the check for 
representativeness and the effects 
of practice, we were able to 
proceed with the performance 
comparisons of 1993 AOT students 
and students from 1989 and 1985. 
When making these comparisons, 
student results from 1989 and 
1985 were excluded from the 
analysis if those students were 
from schools that were smaller 
than the smallest school taking 
part in the 1993 AOT study. 


Ali comparisons made in this 
study (i.e., to check for 
representativeness and the effects 
of practice, and to make 
judgements about achievement 
over time) were assessed using 
statistical tests of significance. 
Because these tests are very 
sensitive when used with large © 
samples, as was the case in this 
study, only differences having a 
probability less than 0.01 were 
considered to be significant. 


Results 

Table 8-2 presents a comparison of 
AOT sub-group results and 1993 
provincial results. The purpose of 


this comparison was to verify that 
the students selected to participate 
in the special study were 


representative of students 
throughout the province. 


Table 8-2 
Comparison of AOT Sub-group Results with 1993 Provincial Results* 


AOT 93 Province AOT 89 Province AOT 85 Province 
1993 Items 1993 Items 1993 Items 1993 Items 1993 Items 1993 Items 
et eee ees, (eee it 
Grade Level 3 Language Learning , 
Number of 1993 Questions 40 40 19 19 19 19 
Raw Score Mean 26.3 26.8 13.4** 13.0 13.3 13.2 
Standard Deviation 7.9 79 3.8 3.9 4.1 4.0 
Number of Students 959 33 546 935 33 546 938 33 546 
Grade 6 Social Studies ; 
Number of 1993 Questions 47 47 20 20 22 22 
Raw Score Mean 29.8 29.9 12.3 12.3 12.8 13.0 
Standard Deviation 7.2 7.3 3.2 3.3 3.8 3.8 
Number of Students 938 28 987 932 28 987 935 28 987 
Grade 9 Science 
Number of 1993 Questions 65 65 45 45 42 42 
Raw Score Mean 40.4 41.2 277" - 28.7 25.8"° 26.5 
Standard Deviation 10.6 10.4 7.6 7.4 7.3 7.1 
Number of Students 986 25 178 977 25 178 972 25 178 


*Students from schools smaller than the smallest school taking part in the AOT study are excluded from this analysis. Provincial data in this 
table, therefore, differ from those reported elsewhere in this report. 


**On these items, the mean of the AOT group is significantly different from the provincial mean. 


The data indicate that there were 
no significant differences between 
provincial means and the means of 
the Grade 6 Social Studies AOT89 
and AOTS85 groups, or between 
provincial means and the Grade 
Level 3 Language Learning AOT85 
means. There were, however, 
significant differences between the 
provincial mean and the AOT89 
group in Grade 3. As well, in Grade 
9, both the AOT89 and the AOT85 
group means differed significantly 
from the provincial means on the 
same grouping of items. This 


indicates that the Grade 6 Social 
Studies sample and the Grade 3 
AOTS85 sample were representative 
of the province. However, the Grade 
3 1989 AOT group and the Grade 9 
1989 and 1985 AOT groups were 
not. To address this issue of non- 
representativeness, test equating 
procedures were used when 
comparing 1993 Grade 3 
performance with that in 1989 and 


’ when comparing 1993 Grade 9 


performance with that from 1989 
and 1985. 


The AOT comparison data is 
provided in Table 8-3. Included in 
this table, for each grade, is the 
number of questions on which 
comparisons were based, the mean 
and standard deviation for each 
year of comparison, and the 
number of students in each 
comparison group. It should be 
noted that equated means are 
reported for the Grade 3 1989 AOT 
group and the Grade 9 1989 and 
1985 AOT groups. 
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Table 8-3 
Achievement-Over-Time Comparison of Descriptive Statistics* 


Grade Level 3 Language Learning 
Number of Questions 

Raw Score Mean 

Standard Deviation 

Number of Students 


Grade 6 Social Studies 
Number of Questions 
Raw Score Mean 
Standard Deviation 
Number of Students 


Grade 9 Science 
Number of Questions 
Raw Score Mean 
Standard Deviation 
Number of Students 


AOT 89 Group | 1989 Province AOT 85 Group 1985 Province 
(Wrote in 1993) | (Wrote in 1989) | (Wrote in 1993) | (Wrote in 1985) 

21 21 21 21 

16.1" 15.3 15.3" 14.6 

4.4 4.5 4.0 3.7 

935 31014 938 30 976 

30 30 28 28 

20.2** 19.4 18.2** 17.2 

6.0 5.8 4.6 48 

932 29 554 935 28 793 

20 20 23 23 

13.3 13.6 13.6" 14.0 

4.0 3.8 4.1 4.2 

977 27 041 972 29 513 


* Students from schools smaller than the smallest school taking part in the AOT study are excluded from this analysis. Provincial data from 
1989 and 1985 differ, therefore, from those reported in earlier reports. 


** On these items, the mean of the AOT group is significantly different from the provincial mean. 


ee ee 


The data indicate that 1993 
students in Grade Level 3 and 
Grade 6 achieved significantly 
higher scores than did their cohorts 
from earlier years. In Grade 9 the 
trends were reversed, with the 
students from earlier years 
achieving the higher scores. 


Inferences about these differences 
should be drawn with caution, as 
the results of the practice-effects 
portion of the study indicate that 
some of the 1993 scores may be 
spuriously high due to the effect of 
students having prepared for the 
test using items from the 1989 and 
1985 achievement tests. In 
particular, there is some evidence 


8&8 


to suggest that the Grade 3 AOT85 
and the Grade 6 AOT89 groups 
may have benefited from such 
practice. Consequently, it seems 
reasonable to infer that some, 
albeit not all, of the score increases 
in grades 3 and 6 are due to the 
effects of practice. 


The 1993 decline in Grade 9 scores, 
relative to those from 1989 and 
1985, does not seem to have been 
mitigated by the effects of practice: 
few Grade 9 AOT students were 
reported to have used previous 
tests in preparation for the 
achievement test, and those who 
did practice did not achieve higher 
scores. 


In conclusion, the data from the 
multiple-choice study indicate that 
achievement in Grade Level 3 
Language Learning and Grade 6 
Social Studies is somewhat higher 
than that in earlier years. Science 
achievement in Grade 9 does not 
seem to have fared as well, 
however, with 1993 students 
scoring about the same as students 
in 1989, but lower than those in 


1985. 


Written-Response Study 
Grade Level 3 

Language Learning 
Design and Methodology 


Students’ writing performance has 
been of particular interest to 
educators and to the public since 
the beginning of the Achievement 
Testing Program. Because of this 
interest, a study was initiated to 
compare 1993 writing with 1989 
writing and 1985 writing in Grade 
Level 3 Language Learning. This 
was a descriptive study that 
required teacher-readers to take a 
research or reader-as-observer look 
at the papers they read, rather 
than the usual evaluative or 
reader-as-assessor view of a 
teacher-marker. No attempt was 
made to rescore papers; rather, 
teacher-readers described features 
of the 1993, 1989, and 1985 writing 
in the three scoring categories: 


Content and Development 
-relationship between events, 
actions, and/or ideas and the 
context established by the 
writer 
—beginning 
—specificity of detail 


~connections and/or 
relationships between events, 
actions, details, and or 
characters 

~ending 

~reader-writer relationship 


° Use of Language 

—degree to which standard 
sentence constructions are 
present and controlled 

—presence of different sentence 
patterns and length 

—quality of words and 
expressions 


eConventions 
—end punctuation and 
capitalization 
—spelling 


—clarity 


Comparisons were made at two 
standards: Clearly Meets the 
Acceptable Standard (formerly 
Acceptable—3), which represents 
work at an acceptable level for 
students completing Grade 3, and 
Clearly Meets the Standard of 
Excellence (formerly Excellent—5), 
which represents outstanding work 
for students completing Grade 3. 


Papers read in the study were 
selected at random from papers 
that received scores of Clearly Meets 
the Acceptable Standard or Clearly 
Meets Standard of Excellence on the 
June 1993, June 1989, and June 
1985 achievement tests. 


A group of experienced teacher- 
readers, representing all major 
regions of the province and a 
variety of school settings, reviewed 
the selected papers. 


Working alone, then in pairs, and 
then discussing the papers as a 
group, the teacher-readers 
described the papers for features of 
content and development, use of 


_language, and conventions. 


They then compared their 
descriptions of 1993, 1989, 1985 
Clearly Meets the Acceptable 
Standard (3) and Clearly Beyond. 
Standard of Excellence (5) papers 
to draw conclusions. 
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Results 


Clearly Meets the Acceptable 
Standard (3) 

Teacher-readers felt that papers 
judged to Clearly Meet the 
Acceptable Standard (3) in 1993 
were not significantly different 


from 1985 Acceptable (3). Although 
they thought that 1989 Acceptable 
(3) papers were in the range of the 
1993 papers Clearly Meeting the 
Acceptable Standard, they found 
both the 1993 and the 1985 papers 
to be generally superior to the 1989 


Clearly Meets the Acceptable Standard (3) in Grade Level 3 Language Learning 


papers, particularly in content and 
development. Teacher-reader 
comments on key features of these 
papers are summarized below. 


Key Features of 1993 Papers Key Features of 1989 Papers 


Key Features of 1985 Papers 


Content and Development Content and Development 


Content and Development 


’ events, actions, and/or ideas appropriate 
for context 


‘only some events, actions, and/or ideas are 
appropriate for context 


‘clear idea of storyline development ‘disjointed storylines 


sstrong beginnings ‘confusing beginnings 


*middle consists of actions, (events), 
characters not directly related 


‘events were rambling and connections to 
other events or characters were not evident 


events actions, and/or ideas are 
appropriate for context 


*sequential development; simple recounting 
of events 


‘appropriate beginnings 


«middle consists of actions, (events), 
characters not directly related 


«weak or missing transitions were common in all three years 
‘appropriate but abrupt closures were common in all three years 


Use of Language Use of Language 


Use of Language 


standard sentence constructions ‘standard sentence constructions not 


present 
variety of sentence length and type 


specific words attempted 


usually present 
‘simple sentences predominate 


‘few vocabulary risks 


*standard sentence constructions present 


*some variety of sentence length and type 


‘ew vocabulary risks 


simple subordination or coordination using (and/or) common in all three years 


Conventions 


Conventions 


Conventions 


*somewhat controlled (many run-on 
sentences 


‘end punctuation: periods or 
exclamation marks 


‘quotation marks attempted 


spelling errors common; 
errors relatively phonetic 


somewhat controlled 


send punctuation: periods 


‘quotation marks generally not attempted 


‘ew spelling errors 


ery controlled, confidently used 
send punctuation: periods 


‘quotation marks generally not attempted 


‘few spelling errors 
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Excellence (5) 

Teacher-readers felt that papers 
judged to be clearly beyond the 
Standard of Excellence (5) in 1993 


were superior to the papers from 
1989 and 1985, which were, 
nonetheless, high quality but more 
similar to each other. A summary | 


Clearly Beyond the Standard of Excellence (5) In Grade Level 3 Language Learning 


of the teacher-reader comments on 
differences and similarities in the 
three reporting categories follows. 


Key Features of 1993 Papers 


Content and Development 


Key Features of 1989 Papers 


Key Features of 1985 Papers 


Content and Development 


Content and Development 


events, actions, and/or ideas consistently 
appropriate 


‘beginning captures reader’s attention; 
provides direction for writing 


sspecific, effective details consistently used 


sconnections/relationships between events, 
actions, details and/or characters 
consistently maintained 


‘clearly wrote for an audience 
“strong, connected endings 


«writer's voice was strong 


sappropriate ideas and details used to 
develop storylines 


strong beginning 


specific details used 


events and details were arranged in a 
purposetul, effective order 


audience not clear 
sabrupt and/or contrived endings 


writer's voice not evident 


‘appropriate ideas and details used to 
develop storylines 


*strong beginning 


“specific details used 


events and details were arranged ina 
purposeful, effective order 


audience not clear 
abrupt and/or contrived endings 


ewriter’s voice not evident 


sstudents in all three years captured reader interest by writing with imagination and creativity 


Use of Language 
‘presentation consistently controlled 


"good variety of sentence type and length 


ewords and expressions timely and 
consistently precise and effective 


willingness to take vocabulary risks with 
good results 


presentation usually controlled 


some variety of sentence type and length 


specific or descriptive verbs tended to be 
well used 


willingness to take vocabulary risks, but 
these were not always successful 


Use of Language Use of Language 


‘presentation usually controlled 


*some variety of sentence type and length 


*specific or descriptive adjectives and verbs 
tended to be used well 


ew vocabulary risks taken 


Conventions 


Conventions 


Conventions 


‘punctuation controlled in all three years 
‘few spelling errors in all three years 


Concluding Comments 


The question that guided this study 


was: Has writing by Grade Level 
3 Language Learning students 
improved from 1985 to 1993? 


Teachers participating in the study 


thought that in 1993 students 
clearly meeting the acceptable 
standard (3) produced writing of 
similar quality when compared to 


their 1985 and 1989 counterparts. 
However, in 1993 students, whose 
work was clearly beyond the 
standard of excellence, produced 
higher calibre writing when 
compared to their 1985 and 1989 
counterparts. 


Teacher-readers made the 
following observations related to 


the changing nature of the writing 
over the three test years. 


The different prompts used on the 
three tests may have influenced 
the type of writing produced by 
students. Teacher-readers felt that 
the 1985 prompt tended to be 
narrow in focus and did not allow 
the students enough leeway to 
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express themselves. The prompt in 
1989 was perhaps more open- 
ended, but much of the students’ 
interpretation hinged upon their 
understanding of the word 
“fragile”, which figured 
prominently on a large packing box 
that fell from a fast-moving truck. 


The teacher-readers all agreed that 
the 1993 “hats” prompt gave 
students the most latitude to 
develop their stories and that the 
picture of different hats, which 
accompanied the prompt, gave 
them possibilities from which to 
springboard their ideas. 


All believed that the 10 minutes of 
collaboration time given in 1993 
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before writing commenced made a 
difference to the students, if not 
directly in their writing, at least in 
their comfort level. The fact that 
students were told to choose the 
format—story, letter, or series of 
diary/journal entries—that would 
allow them to do their best writing 
was also a positive change in the 
teachers’ view. 


It was very obvious to the teachers 
that the students writing in 1993 
come from print-rich classroom 
environments where literature 
plays a key role. This strong 
literature base was very evident in 
the writing that these students 
produced. 


While not in the realm of 
assessment, teachers also noted 
that penmanship quality has 
deteriorated over the three test 
years. As well, in the writing from 
all three years, both themes and 
supporting details strongly 
reflected the popular media 
influences of the time. Storylines 
and incidents from popular 
television shows, movies, cartoons, 
and even commercials were often 
interwoven in the students’ 
writing. 


Written-Response Study 
Grade 6 Social Studies 
Design 


Students’ writing performance has 
been of particular interest to 
educators and to the public since 
the beginning of the Achievement 
Testing Program. As a consequence 
of this interest in how well 
students write, a study was 
initiated to compare 1993 writing 
with 1989 writing and 1985 
writing in Grade 6 Social Studies. 
This was a descriptive study that 
required the teacher-readers to 
take a research or reader-as- 
observer stance toward the papers 
they read. Teacher-readers 
described features of the 1993, 
1989, and 1985 writing in two 
scoring categories: 


State and Support an Opinion/ © 
Position 
© Evidence of position 
*Reasons/arguments used to 
support the position taken 
® Quality of examples to support 
the position taken 


Quality of Language and 
Expression 
° Organization 
-focus 
—coherent order 
—connections between main 
ideas and/or detail 


*Vocabulary and Conventions 
-effective words and 
expressions 
—correct grammar, punctuation, 
spelling, capitalization, 
sentence structure 
—effective communication 


In 1985 and 1989, the written- 
response assignment involved short 
answers and one extended response. 
In 1993, the assignment was limited 
to one extended response. In order 
to facilitate comparison of similar 
assignments, only the extended 
response from 1985 and 1989 was 
compared with the 1993 
assignment. 


Comparisons were made at two 
standards: Acceptable (3), which 
represents work at an acceptable 


level for students completing 
Grade 6, and Excellent (5), which 
represents outstanding work for 
students completing Grade 6. 


Methodology 

Papers read in the study were 
selected at random from papers 
that received scores of Acceptable 
(3) or Excellent (5) on the June 
1993, June 1989, and June 1985 
achievement tests. 


A group of ten experienced teacher-, 
readers, representing all major 
regions of the province and a 
variety of school settings, reviewed 
the selected papers. 


Working alone and in pairs, and 
then discussing papers as a group, 
the teacher-readers described the 
papers for features of state and 
support an opinion/position and 
quality of language and expression. 


They then compared their 
descriptions of 1993, 1989, and 
1985 Acceptable (3) and Excellent 
(5) papers to draw conclusions. 
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Results 


Acceptable Standard (3) 
Teacher-readers felt that the 
papers judged to achieve the 
Acceptable Standard (3) were 
slightly better in 1993 than in 1989 
and 1985. Teacher-reader 
comments of these papers are 
summarized below. 


Acceptable Standard (3) in Grade 6 Social Studies 


Key Features of 1993 Papers 


Key Features of 1989 Papers Key Features of 1985 Papers 


State and Support an Opinion/Position 


Persuasiveness and Logic of Supporting Persuasiveness and Logic of Supporting 
Arguments Arguments 


‘identifiable position presented 


‘ideas presented but not developed 
“use of implicit rather than explicit ideas 


4imited defence of position, supported 
by generalities 


position supported by personal experiences 


*some opinions reflect extremes found in 
media today 


‘identifiable position presented ‘identifiable position presented 


‘mechanical argumentation 


‘ideas presented but not developed ideas presented but not developed 
suse of implicit rather than explicit ideas “use of implicit rather than explicit ideas 
‘limited defence of position, supported ‘limited defence of position, supported 
by generalities , by generalities (not always accurate) 


position supported by personal experiences ‘position supported by personal experiences 
: (sometimes presented as fact) 


‘limited development, use, and dimited development, use, and ‘limited development, use, and 
understanding of social studies concepts understanding of social studies concepts understanding of social studies concepts 
‘limited understanding of levels of 

government ° 
Quality of Language and Expression Quality of Language and Expression Quality of Language and Expression 
*paragraphing skills evident, use of ‘poor organization and structure _ poor organization and structure 

closing paragraph 

‘frequent errors in grammar, punctuation, ¢requent errors in grammar, punctuation, ‘frequent errors in grammar, punctuation, 
spelling, and sentence structure spelling, and sentence structure Spelling, and sentence structure 
language general not specific ‘language general not specific ‘language general not specific 

specific words attempted dimited vocabulary ‘limited vocabulary 
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Excellent Standard (5) 
Teacher-readers felt that papers 
judged to achieve the Excellent 
Standard of Excellence (5) were 
superior to papers from 1989 and 


1985. A summary of the teacher- 
reader comments on differences 
and similarities in the two 
reporting categories follows. 


Excellent Standard (5) in Grade 6 Social Studies 


Key Features of 1993 Papers 


Key Features of 1989 Papers 


Key Features of 1985 Papers 


State and Support an Opinion/Position 


‘strong, clear position is taken 


position supported by one or more ideas 


‘convincing arguments well developed 
‘consistent position throughout 


‘logical sequence to arguments that are 
based on cause/effect relationships 


*solid understanding of social studies 
concepts demonstrated throughout 


‘good understanding of levels of 
government 


Quality of Language and Expression 


writing fluent and well organized 
paragraphs containing a main idea and 
supporting facts 


‘ew minor errors in conventions 


vocabulary is accurate and effective 


Persuasiveness and Logic of Supporting 


Arguments 


‘clearly identifiable position taken 


‘position supported by one or more relevant 
ideas 


*sound arguments developed 
sarguments lack consistency 


supportive examples are specific and 
relevant 


*social studies concepts used 


writing generally fluent and good paragraph 
development 


"some minor errors in grammar, punctuation, 


spelling, and sentence structure 


‘language is appropriate and specific 


Persuasiveness and Logic of Supporting 
Arguments 


‘identifiable position presented 
“position supported by one or more 
arguments (personal experiences) 
arguments not fully developed 
arguments lack consistency 


‘examples very specific 


‘limited inclusion of social studies concepts 


Quality of Language and Expression Quality of Language and Expression 


‘logical organization but weak or missing 
conclusion 


limited number of errors in grammar, 
punctuation, spelling, and sentence 
structure 


‘language is general not specific 
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Concluding Comments 


The question that guided this study 
was: Has writing by Grade 6 
Social Studies students 
improved from 1985 to 1993? 
Teachers participating in the study 
thought that, in 1993, students 
achieving the acceptable standard 
(3) produced writing of similar 
quality when compared to their 
1985 and 1989 counterparts. They 
thought that 1993 students 
achieving the excellent standard 
produced higher calibre writing 
when compared to their 1985 and 
1989 counterparts. 


A major change was made to the 
writing component of the 1993 
achievement test. In 1998, students 
addressed only one written- 
response question, whereas in 1985 
students answered four short 
written-response questions before 
attempting the extended response, 
and in 1989 students answered six 
short written-response questions 
before attempting the extended 
response. In addition, students in 

' 1993 had more time to answer the 
extended-response question. 
Within the confines of a test 
situation, some students in 1985 
and 1989 may have experienced 
oversaturation with the test as well 
as lack of time before attempting 
the extended response. 
Teacher-readers made the 
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following observations related to 
the changing nature of the writing 
over the three test years. 


The teacher-readers felt that the 
1993 issue was more general and 
abstract than the 1985 and 1989 
issues for the written-response 
question. They felt that while the 
additional time students had to do 
the writing assignment produced 
better writing than in 1985 and 
1989, a more specific and concrete 
question might have provided 
students a better opportunity to 
achieve higher levels. 


The teacher-readers thought that 
the different prompts used on the 
three tests may have influenced 
the type of writing produced by 
students. In 1985 and 1989, the 
short-answer questions were 
prompts in themselves. The 1993 
prompt was more detailed than the 
others and included visuals. 
However, the teacher-readers 
indicated that the writing prompt 
may have been misleading and 
confusing for students not 
achieving provincial standards. It 
may have been difficult for some 
students to relate the information 
provided in the prompt to the 
assignment, and it may have 
served as a distraction rather than 
as a help to students in their 


writing. The teacher-readers felt 
that this may be true especially for 
students marginally achieving at 
the acceptable standard or lower. 
They felt that higher achieving 
students were better able to use 
the prompt. 


In addition, the teacher-readers 
thought that the 1993 assignment 
limited student choice, as the 
information stated that “there are 
two positions to this issue” and did - 
not state that students could take 
a qualified position. However, this 
did not prevent a number of 
students from taking qualified 
positions in their writing. 


Teachers also indicated that, in 
some cases, students’ planning did 
not relate to their writing. Once 
students began writing, they either 
forgot about the plan or found 
going back and forth to the 
planning pages inconvenient. This 
is common practice for students. 
The teacher-readers suggested that 
a tear-out sheet should be provided 
for the written response so that 
students could make better use of 


planning. 


_ Section 9 


Age 9 Achievement 


Methods of improving the current 
assessment program have been 
explored over the last few years. 
One of the methods under 
investigation has been to assess a 
particular age group of students 
rather than particular grade levels. 
In 1993, in order to collect age- 
based information about student 
achievement in Language 
Learning, a special study to sample 
nine-year-olds in grades 2, 3, and 4 
classrooms was undertaken. 


Key questions that guided the 

design of this study included: 

© What level of performance do 
nine-year-old students 
demonstrate in reading and 
writing? 

© How do levels of performance 
compare between students of the 
same age but different grades in 
school? 

° What are some of the 
administrative concerns related 
to age-based assessment? 

* Do teachers support age-based 
assessment? 


Design 

Both the reading and writing 
components of the Age 9 
assessment were the same as the 
Grade Level 3 Language Learning 
assessment. 


The written component was 
marked using a levels-based 
scoring guide. The reporting 
categories (Content and 
Development, Use of Language, 
and Conventions) were the same as 
those used for the regular Grade 
Level 3 assessment. However, the 
five scales were as follows: 

¢ May Be Beyond Grade Level 4 
* Grade Level 4 

* Grade Level 3 

© Grade Level 2 

¢ Not Yet At Grade Level 2 


The specific criteria outlined in 
each level were directly related to 
the descriptors found at the 
various levels in the Language 
Learning component. A sample of 
papers from students aged 9 and in 
Grade Level 3 were included in the 
design. The papers were marked a 
second time for this study using 


. the levels-based scoring guide. 


Methodology 

A total of 1 032 students from 29 
schools participated in the Age 9 
study. Schools were chosen 
randomly and reflected a 
provincially representative sample. 


_The number of students from 


grades 2, 3, and 4 classrooms in 
the Age 9 sample was in the same 


proportion as the number of Age 9 
students in each grade for the 
province overall. 


School principals were contacted by 
phone to confirm administrative 
procedures and to collect an accurate 
count of students in their school who 
would be nine years of age and not in 
Grade Level 3 on the date the 
assessment was administered. 


Both the reading and writing 
components of the Age 9 assessment 
were administered at the same time 
and followed the same 
administration procedures as the 
Grade Level 3 Language Learning — 
assessment. 


Results 

The findings of this study are 

discussed in conjunction with the 

key questions that guided the project 

design: 

¢ What level of performance do 
nine-year-old students 
demonstrate in reading and 
writing? 


The students’ performance in - 
reading and writing was scored 
according to level of performance. 
Provincial results are shown in 
tables 9-1, 9-2, and 9-3. 
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Table 9-1 


Age 9 Study, Language Learning 
Percentage of Students Achieving Level of Performance on Both Components of the Test 
June 1993 


Level of Performance* Reading Component Writing Component 
May Be Beyond Grade Level 4 18.7 5.9 
Grade Level 4 25.6 20.0 
Grade Level 3 31.6 35.9 
Grade level 2 16.5 29.0 
Not Yet At Grade Level 2 7.6 9.1 


“Level of performance in this context refers to one of the seven grade levels outlined in the Language Learning component of the 
Program of Studies (revised 1993). 


The highest percentage of students with fewer students demonstrating performance level of Age 9 
in the Age 9 sample demonstrated performance at Grade Level 2 and students was higher in reading - 
performance at Grade Level 3, Grade Level 4. Overall, than in writing. 


Table 9-2 
Age 9 Study, Language Learning 
Percentage of Students Achieving Level of Performance by Grade 


Reading Component 
June 1993 
; May Be 
Number of Not Yet at | Grade | Grade| Grade | Beyond Grade 


Students | Grade Level 2} Level 2} Level 3| Level 4 Level Total 
TBs ere. fcc. 15.4 0.08) 0.0/2) 

1 824 

956 


Table 9-3 
Age 9 Study, Language Learning 
Percentage of Students Achieving Level of Performance by Grade 


Writing Component 
June 1993 
May Be 
Number of Not Yet at Grade | Grade | Grade | Beyond Grade 
Grade _Students 


Level 2 |Level 3| Level4| Level4 _|_ Total 


eo eiae = “g8 | 00 | oo | oo | 1000 
eee ee eee ee 32.3. 16.6 | 32 100.0 
rs Re pce ee 258" P26 B BBL F000, 
TOTAL 956 . 29.0 20.8 6.1 100.0 
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* How do levels of performance 
compare between students of 
the same age but different 
grades in school? 


Tables 9-2 and 9-3 indicate that 
student performance does appear to 
be related somewhat to grade level; 
that is, nine-year-olds in Grade 
Level 4 demonstrated higher levels 
of reading and writing than nine- 
year-olds in Grade Level 3. 
Similarly, nine-year-olds in grade 
levels 3 and 4 performed at higher 
levels than nine-year-olds:in Grade 
Level 2. However, a high percentage 
of students still did not achieve an 
acceptable level of performance for 
the grade they were in. Of students 
in Grade 2 classrooms, 
approximately 62% did not achieve 
at Grade Level 2 in reading and 
69% did not achieve at Grade Level 
2 in writing. Of Grade 3 students, 
33% did not achieve at grade level 
in reading and approximately 44% 
did not achieve at grade level in 
writing. Approximately 45% of the 
students in Grade 4 classrooms did 
not perform at grade level in 
reading; 66% of them achieved 
below grade level in writing. 


It is unclear whether the 
differences among grades are due 
to promotion policies and the type 
of instruction provided for students 
who are retained, to the number of 
years students have been in school, 
to grade-specific curricula being 
taught, or to other factors. 


¢ What are some of the 
administrative concerns 
related to age-based 
assessment? 


Findings from the administration 
of the Age 9 assessment showed 
few reactions to the concept of age- 
based assessment. Only one 
principal expressed a concern 
about administering the 
assessment to only some students 
from classes organized by grade. 


* Do teachers support age- 
based assessment? 


The 150 teachers who marked the 
Grade Level 3 compositions were . 
asked if they would support 
provincial assessments for nine- 
year-olds instead of for Grade 
Level 3 students. Only 23% of 


those teachers were in favour of a 
move toward age-based 
assessment. Most expressed 
concerns about pulling a sample of 
students away from their regular 
classes and teachers. 


Teachers who marked the Age 9 
writing component were generally 
positive about the levels-based 
scoring guide that was used. 


Concluding Comments 
Through this study, we learned 
that it is possible to conduct age- 
based assessment. However, the 
approach does not correspond to 
the administrative structure that 
currently exists in schools. 
Administration of age-based 
assessment, given the nature and 
purpose of the provincial 
Achievement Testing Program, can 
be disruptive, and the assessment 
does not provide teachers, 
administrators, and parents with 
information that is most useful to 
them. These findings will 
contribute to the planning and 
structuring of future assessments. 
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Appendix A. 


Standards 


The move toward a results-based 
curriculum has re-emphasized the 
need for a clear delineation of 
standards. and their purpose. 

All standards and all methods 

of setting standards require 
judgment. 


The process of setting a 
standard can be only as good as 
the judgments that go into it. 
The standard will depend on 
whose judgments are involved 
in the process. In this sense, all 
standards are subjective. Yet, 
once a standard has been set, 
the decisions based on it can be 
made objectively. Instead of a 
separate set of judgments for 
each test-taker, you will have 
the same set of judgments 
applied to al] test-takers. 
Standards cannot be objectively 
determined, but they can be 
objectively applied. 


Definitions 

The Achievement Testing Program 
is directly concerned with three 
different but related standards. 
These provincial standards are 
curriculum standards, achievement 
standards, and assessment 
standards. 


¢ Curriculum Standards are 
the expected student learnings 
sequenced into grade levels. 
They include broad statements 
of knowledge, skill, and attitude 
expectations against which 
student performance is judged. 
These standards are established 


1 Passing Scores; Samuel A. Livingston, Michael J. Zieky; Educational Testing Service, 1982. 


in the process of curriculum 
development and are found in 
the Program of Studies 
document produced for each 
course. 


Achievement Standards are 
judgments that specify what 
percentages of students are 
expected to achieve an 
acceptable and an excellent level 
of performance in relation to 
each course of studies, i.e., the 
relevant curriculum standards. 
It is important to point out that 
this judgment is not a prediction 
of the percentage of students 
who will actually achieve 
acceptable or excellent levels of 
performance but rather is a 
specification of the percentage of 
students at a given age or grade 
in school who are expected to 
achieve an acceptable or 
excellent level. These standards 
apply to school, jurisdiction, and 
provincial performance. 


Assessment Standards are the 
criteria adopted for judging 
actual student achievement 
relative to curriculum standards. 
They are ultimately expressed 
and applied as test scores. They 
are derived from answers to 
questions such as: 


What scores must a student 
obtain or how many 
questions on a given test 
must a student answer 
correctly in order for his/her 


performance on the test to be 
judged as acceptable or 
excellent? 


These standards apply to 
individual student performance. 


Indirectly, the Achievement 
Testing Program influences local 
targets. 


¢ Local Targets are the objectives 
set in schools and jurisdictions to 
assist students in moving toward 
or exceeding the provincial 
assessment standards. These 
local targets reflect the specific 
needs of individuals and groups 
within a specific community. 


The Student Evaluation Branch 

is responsible for establishing 

and reporting the provincial 
assessment standards and shares 
responsibility with the Curriculum 
Branch for establishing provincial 
achievement standards as they 
relate to the Achievement Testing 
Program. 
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Purpose 


The purpose of defining standards, 
and the subsequent process of 
setting the provincial assessment 
standard within the sphere of the 
Achievement Testing Program, is 
to answer questions such as: 


* What is acceptable and 
excellent performance in 
relation to the curricular 
expectations for students at. 
the age or grade being tested? 


What percentages of students 
at the age or grade being tested 
ought to achieve an acceptable 
and excellent level, assuming 
adequate teaching and 
resources? 


© What scores on a specific test 
shall reasonably represent 
acceptable and excellent 
performance respectively? 


What are the prevailing 
strengths and weaknesses of 
Alberta students in relation to 
the curriculum being tested? 
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In essence, for each test and level 
of performance, the assessment 
standards setters are challenged 
to answer the question: 


What score must a student 
obtain or how many questions 
must a student answer 
correctly to be judged as 
having achieved an acceptable 
or excellent standard? 


Satisfactory performance is 
demonstrated when the percentage 
of students scoring at or above the 
assessment standards is equal to 
or greater than the achievement 
standards. 


In this report, tables in sections 2, 
3, 4, and 9 dealing with standards 
show what percentage of students 
achieved standards on each major 
component and on the total test. 


The Assessment 
Standard-Setting 
Process 


Figure A-1 shows the model 
established in 1991 by the Student 
Evaluation Branch to set 
standards for achievement tests. 
The objective was to widen the 
process of setting assessment 
standards as much as possible 
over previous years and 
especially to provide for 
community input and 
feedback. The process involves 
individuals and groups making 
judgments that contribute to 
establishing the assessment and 
achievement standards. 


Except for the members of the 
Public Advisory Committee, 

the members of all committees 
outlined in Figure A-1 are expected 
to be highly knowledgeable about 
both the curriculum and the 
learning characteristics of the 
students who are writing the tests. 


Figure A-1 
Process Model for Standard Setting 
Achievement Testing Program 


June 1993 


2 


1 
Curriculum and 
Test Developers 
Committee 


Test Review 
Committee 


INPUT 


3 
Public Advisory 
Committee 


6 
Director’s 
Final Standards Review 
Committee — 
Representatives 
from 1, 2,3, 4,5 


PRODUCT 


ASSESSMENT STANDARDS 
Raw scores (cut-off scores) representing 
Acceptable Standard and 


Standard of Excellence 
and 
ACHIEVEMENT STANDARDS 
Percentage of students expected to 
achieve or exceed Assessment Standards 


Teacher Standards 
Committee 


5 
Analytic Services 
Committee 
Student Evaluation 


1.Curriculum and Test 
Developers Committee 
There is one Curriculum and 
Test Developers Committee for 
each achievement test. The 
committee consists of Alberta 
Education consultants, 
curriculum developers, and 
test development specialists. 
Their objective is to recommend 
assessment standards to the 
Final Standards Review 
Committee. They may also 
review and make 
recommendations on 
Achievement Standards. 


2.Test Review Committee 
There is one Test Review 
Committee for each achievement 
test. Experienced and 
knowledgeable educators— 
teachers, superintendents, 
university professors, and 
Alberta Education consultants— 
meet with the developers of 
each test. Their purpose is to 
review the test and make 
recommendations for 
improvements where necessary. 
In addition, the committee 
reviews the appropriateness of 
current course achievement and 
assessment standards. 


3. Public Advisory Committee 
A Public Advisory Committee, 
composed of representatives from 
the general public, met again in 
1993 and discussed standards. 
See Appendix B for more 
information on the proceedings 
of this committee. 


4. Teacher Standards 
Committee 
Approximately 20 experienced 
teachers from different areas of 
the province are selected to sit on 
a Teacher Standards Committee 
for each test. To be selected for a 
committee, a teacher must have 
been teaching in the grade and 
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subject area for the previous two and statistics for each test. A committees and is chaired by the 


years. At present, the procedures critical function is to ensure that Director of Student Evaluation. 
used to aid teachers in setting Alberta Education’s standard- In separate sessions, the 
the assessment standards are setting procedures produce recommendations of all test 
the modified Angoff method for technically valid results that committees are presented and, 
Mathematics and Science meet the strict requirements of through consensus, final 
subjects and the Nedelsky high quality professional studies. Assessment Standards are 
method for Humanities subjects.” The committee makes adopted for each test. 
The teachers make judgments recommendations for assessment 
about appropriate assessment standard-setting procedures,. 
standards and recommendations leads individual group 1993 Assessment and 
: the ee Eos Review discussions, and hier a Achievement Standards 
ommittee. As well, they review improvements where necessary. 

Achievement Standards for Another function is to determine, Tables A-1 to A-3 show the 

: : assessment and achievement 
appropriateness. independently, assessment é 

standards through statistical standards adopted in June 1993 
5.Analytic Services Committee analyses of current student by the F inal Standards Review 

The Analytic Services Unit of the achievement data. Committee for the grades 3, 6, 
Student Evaluation Branch is and 9 achievement tests. The 
the professional quality control 6.The Final Standards Review _—‘*@bles also show the percentages of 
and advisory group for the Committee students achieving provincial 
complete process of standard The Final Standards Review assessment standards. 
setting. There is a committee of Committee consists of 
professionals in psychometrics representatives from the above 


2 “A Consumer's Guide to Setting Performance Standards on Criterion-Referenced Tests,” Ronald A. Berk: Review of Educational 
Research, Spring, 1986, Volume 56. 


Table A-1 
Grade Level 3 English Language Learning 
Assessment and Achievement Standards 


June 1993 
Standard of Excellence 
Provincial Provincial 
Assessment* Percentage Achieving Achievement** 
Category Standard Assessment Standard Standard 
. (Raw Score) (%) 
Total Test 79/100 14.1 15 
Writing Skills 15/20 14.5 15 
Reading Skills 33/40 28.8 15 
Acceptable Standard 
Provincial Provincial 
Assessment* Percentage Achieving Achievement** 
Category Standard Assessment Standard Standard 
(Raw Score) (%) 
Total Test 48/100 82.2 85 
Writing Skills 9/20 81.3 85 
Reading Skills 19/40 82.2 85 


“The Provincial Assessment Standard is a score determined by appropriate standard-setting procedures and is the lowest score a 
student must achieve for his/her performance to be judged acceptable or excellent in relation to curricular expectations. 


**The Provincial Achievement Standard refers to the percentage of students expected to meet the Provincial Assessment Standard. 
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Table A-2 


Grade 6 Social Studies 
Assessment and Achievement Standards 
June 1993 
Standard of Excellence 
Provincial Provincial 
Assessment Percentage Achieving Achievement 
Category Standard Assessment Standard Standard 
(Raw Score) (%) 
Tota! Test 80/100 8.0 15 
Skills 50/64 6.2 15 
Knowledge 20/24 28.7 15 
Acceptable Standard 
Provincial : Provincial 
Assessment Percentage Achieving Achievement 
Category Standard Assessment Standard Standard 
(Raw Score) (%) 
Total Test 49/100 80.8 85 
Skills 30/64 77.2 85 
Knowledge 13/24 83.5 85 
i a ee oe eee ee earl 
Table A-3 
Grade 9 Science 
Assessment and Achievement Standards 
June 1993 
Standard of Excellence 
Provincial Provincial 
Assessment Percentage Achieving Achievement 
Category Standard Assessment Standard Standard 
(Raw Score) (%) 
Total Test 52/65 17.1 15 
Skills 23/29 21.5 15 
Concepts 29/36 19.3 15 
Acceptable Standard 
Provincial Provincial 
Assessment Percentage Achieving Achievement 
Category Standard Assessment Standard Standard 
; (Raw Score) (%) 
Total Test 33/65 77.6 85 
Skills 15/29 74.8 85 
Concepts 18/36 80.3 85 
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Appendix B 
: 


Public Review of 
Standards and Results 


On July 21 and 22, 1993, 
representatives of seventeen 
business, professional, and 
community organizations reviewed 
achievement test results and 
discussed standards. This meeting 
was planned to ensure that input 
from groups representing “public” 
interests would be considered in the 
overall reporting of results. The 
purpose of the meeting was to 
gather direct public input into the 
question: 


How well should Alberta 

students achieve in Grade Level 3 
Language Learning, Grade 6 
Social Studies, and Grade 9 
Science? 


The two-day meeting was designed 
to collect information that would 
contribute to a better understanding 
of how well Alberta students are 
doing. This was accomplished by 
exploring and discussing answers to 
four questions. Each question is 
listed below, along with the results 
of the discussions. 


Question: 


What do members of the public 
expect students to know and be 
able to do in Grade Level 3 
Language Learning, Grade 6 
Social Studies, and Grade 9 
Science? 


For each subject area, participants 
listed the knowledge and skills that 
they felt were important learnings 
for students to achieve at each 
grade level. 


The following learnings were 
highlighted by the group as 
important: 


Language Learning 3 
Students should: 


* know how to organize their 
writing using a prompt 
© know how to work cooperatively 


© know the correct use of sentence 
structure, spelling, 
paragraphing, grammar 

* be able to synthesize/draw 
conclusions/make generalizations 


© be able to interpret what they 
read/see 


© be able to follow directions 


© be able to determine general 
word meanings and main ideas 


® be able to relate experience/ 
imagination to the written word 
using appropriate vocabulary. 


Social Studies 6 
Students should: 
° know how government works 


© know the difference between 
needs and wants 


® know how to debate 


* be able to articulate a position 
and present supporting 
arguments 


© be able to organize thoughts 


Science 9 
Students should: 


° know the scientific method 


© know general science concepts 
(facts and relationships) 


© know how science impacts on 
society 


° be able to manipulate formulas 
and variables 


* be able to use process skills 


© be able to apply their 
mathematics background 


© be able to demonstrate scientific 
literacy 


Question: 


How closely do the public 
expectations about student 
learning relate to provincial 
standards? 


Participants reviewed the 1993 
Language Learning, Social 
Studies, and Science tests. There 
was essentially a “match” between 
the public expectations of what 
students should know and be able 
to do and the knowledge and skills 
needed to answer the questions in 
each of the tests. 


Participants were pleased with the 
variety of testing methods 
(numerical response, visuals, 
performance-based assessment) 
used to provide information about 
student achievement. 
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Participants expressed concern 
that the prompt for the Social 
Studies 6 written component may 
have provided students with 
extraneous information and 
therefore confusing rather than 
clarifying the assignment for them. 


Question: 


How well did students actually 
perform on the achievement 
tests? 


Participants identified questions in 
the test that corresponded to 
knowledge and skills they felt were 
most important for students to 
learn. In addition, they discussed 
the percentage of students in 
Alberta that should answer each 
question correctly to meet 
provincial standards. Results 
showing how well students 
actually performed on these 
questions were also presented to 
the group. As well, an initial 
analysis of the composition 
component of the Language 
Learning and Social Studies test 
was presented. 


In each area, students achieved 
some of the expectations 
participants set for specific 
learnings. For other learnings, 
achievement was below 
expectations. These areas are 
listed below. 


Language Learning 3 
Met Standards: 


° be able to use background 
knowledge/experience 


® understand main ideas, key 
details 


© be able to express ideas ina 
coherent written form 
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© be able to draw conclusions, 
make generalizations. 


Below Standards: 


© be able to recognize and 
interpret word meanings 


® be able to make judgements/ 
interpretations 

Social Studies 6 

Met Standards: 

¢ know basic concepts 


© be able to identify geographical 
relationships 


_ © beable to synthesize/interpret 


information 


Below Standards: 


® be able to differentiate between 
facts and opinion 


© be able to evaluate information 


® be able to apply knowledge ina 
global sense 


© be able to take and support a 
clear position in written form. 


Science 9 
Met Standards: 


e know scientific processes (fluid 
systems, buoyancy, gravity) 


© be able to interpret graphs and 
diagrams 


© be able to analyze and evaluate 
given information 
Below Standards: 


e know scientific terms 


.® be able to apply knowledge to 


new situations 


Question: 


Did the group feel that 
students’ performance on the 
achievement tests was good 
enough? 


Language Learning 3 
Generally, participants felt that 
actual student performance met 
what they felt was an acceptable 
standard. 


They did feel, however, that the 
students in the category 
“approaches the acceptable 
standard” should not be forgotten 
about. These are the students we 
need to focus on next to bring them 
up to an acceptable level. Our 
ultimate goal is to have 100% of 
our students achieve the 
acceptable level. 


Social Studies 6 
Most participants did not feel that 
students’ performance is good 
enough. 


The biggest problem seemed to be 
that standards are too low. Also, 
there are not enough students 
achieving the acceptable standard 
or the standard of excellence. 


Science 9 

Committee members were almost 
evenly split between “yes” and “no” 
responses to the questions of 
whether student performance was 
good enough. 


Students generally met the 
committee’s expectations, but 
participants acknowledged that 
there was still room for 
improvement. As well, there were 


differences in personal views 
regarding the level of scientific 
knowledge being assessed at 
Grade 9. 


Other Comments from 
Participants 

Several other comments were 
made. All members of the 
committee had an opportunity to 
provide feedback regarding the 
organization and activities of the 
two-day meeting. Their comments 
are summarized below: 


¢ All participants supported the 
use of a variety of testing 
methods, such as numerical- 
response questions, visuals, and 
performance-based assessment, 
in allowing a greater degree of 
success for more students. 


° Many participants suggested 
that testing should begin at a 
relatively early age so that there 
is time to respond to any 
problems that may show up. 


Participants felt that test scores 
are affected by a variety of 
factors, including attitude 
(motivation) and curriculum 
demands. 


Many participants agreed that 
the Student Evaluation Branch 
needs to continue to address test 
bias, especially in relation to 
Native culture. 


Participants indicated that this 
type of “public meeting” activity 
should continue, possibly on a 
broader scale. 


A Final Note 

The views of participants 
regarding the achievement of 
Alberta students were used to help 


’ get the assessment standards for 


the 1993 achievement tests. 


A more detailed report on the 1993 
meeting was distributed to 
participants and is available upon 
request. Please contact Yvonne 
Johnson at 427-0010 to obtain a 
copy of this report. 


The next meeting for public review 


of achievement standards will be in 
July 1994. 
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Appendix C 


Guidelines for 
Interpreting Results 


Jurisdiction and 

School Reports 

The jurisdiction and school reports 
describe the results achieved by 
students who wrote achievement 
tests in Grade Level 3 Language 
Learning, Grade 6 Social Studies, 
and Grade 9 Science in June 1993. 
These reports include provincial 
results and allow school board 
members, superintendents, 
principals, and teachers to 
compare jurisdiction and school 
results with provincial standards 
and levels of achievement. 


To decide if the achievement of 

their students is “good enough,” 

educators at the school and 

jurisdiction levels can compare 

their results to: 

° provincial standards 

¢ local targets for student 
achievement 

¢ the achievement of students 
throughout.the province of - 
Alberta 

¢ school and jurisdiction results 
from the last provincial 
administration of tests in these 
subjects in 1989 


Use of the Jurisdiction 
and School Reports 


The Achievement Testing Program 
is designed primarily to provide 
group results. However, individual 
student results are provided to 
schools and may be used in 
conjunction with other information 
to plan individual programs for 
students. Also, parents are entitled 


to know how their child did on the 
test. As required by regulation, 
each student’s results must be 
included in the student’s school 
record file. 


The test results are not intended to 

be used as a basis for: 

° making decisions about student 
placement or promotion 

¢ evaluating teacher performance 

® comparing performance between 
or among schools 


The interpretation of results from 
the achievement tests involves 
considering the many factors that 
contribute to achievement. 
Although the tests are designed to: 
assess provincial standards as 
reflected in the general learner 
expectations of the Program of 
Studies, many important aspects of 
learning cannot be measured by 
time-limited, paper-and-pencil 
tests. 


Standards 


Achievement tests and the school 
and jurisdiction reports are 
designed to provide information 
about how well students are doing 
compared to established standards. 
Two standards are set for each 
achievement test. The Provincial 
Assessment Standard is the lowest 
score a student must achieve for 
his or her test performance to be 
judged “acceptable” or “excellent” 
in relation to curricular 
expectations. Standards for 
“excellent” and “acceptable” were 


set for the total test and for the 
two major components of each test. 


Provincial Achievement Standards 
specify the percentage of students 
expected to meet the assessment 
standards. 


Table 2 in the school and 
jurisdiction reports shows the 
assessment and achievement 
standards for the two components 
of each test and for the total test, 
together with the number of 
students who met the standards. 
The table also shows whether the 
number of students in that school. 
or jurisdiction who have achieved 
the standards is significantly 
different from the expected 
number. A difference is reported 
as significant when there is a 5% 
or smaller probability that a 
difference of that size could occur 
by chance. For schools and 
jurisdictions with fewer than five 
students, significance was not 
calculated. 


Although the statistical tests take 
the number of students into 
consideration, results for groups of 
fewer than 25 students must be 
interpreted with particular 
caution. 


Achievement-Over-Time- 
Study 

Some schools participated in a 
study of changes in achievement 
over time. Students in these 
schools were randomly assigned to 
one of three alternative forms of 
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the achievement test for their 
grade. Two forms included items 
from the tests used in the same 
subjects in 1985 and 1989, as well 
as some items from the 1993 test. 
The third form was identical to the 
1993 test. A full explanation of the 
design and the results of the study 
are included in this Provincial 
Report. 


To treat all students in an 
equitable manner, scores on the 
alternative forms were equated so 
that students receive a mark 
equivalent to one they would have 
received if they had written the 
19938 test. This means that the 
results for all students are 
included in most tables. 


For those schools and jurisdictions 
that participated in the study, 
multiple-choice results by 
reporting category and individual 
student subscale scores are 
presented in separate tables for 
each form. 


Also, tables that refer to frequency 
distributions for individual 
multiple-choice and numerical- 
response questions are different for 
schools participating in the study. 
For non-participating schools, 
these tables include distributions 
for the 1993 items only. Provincial 
results in these tables for 1993 
items differ slightly between the 
reports for schools in the study and 
those not in the study. This is 
because the provincial results for 
the non-participating schools are 


students writing the 1993 test, 
whereas for participating schools 
results for 1993 items appearing on 
the 1985 and 1989 forms of the test 
are also used. For schools 
participating in the achievement- 
over-time study, distributions are 
provided for items that appear on 
all forms. 


Results for Individual 
Multiple-Choice 
Questions 

The responses to each item may 
total less than 100% because some 
students did not answer some of 
the questions. 


Grade 6 Social Studies 


Analysis of the student responses 
to multiple-choice items on the 
Grade 6 Social Studies 
Achievement Test revealed 


' problems with five items. Three 


items were deleted from the test 
(22, 33, and 45), and two items (9 
and 49) had two correct answers in 
the possible responses and were 
marked right for both answers. 
None of these items was used for 
the achievement-over-time study; 
thus, the regular form and Form F 
had 47 multiple-choice items and 
forms D and E had 50 multiple- 
choice items. 


Grade Level 3 All forms of the test had the same 
° written-response assignment. This 
Lang uGge Learning section was scored on two scales. 
The written-response section of the To calculate a total score for the 
Grade 3 Language Learning written-response section, the score 


Achievement Test was scored on 
thrée scales. To calculate the total 
score on the written-response 
section, a student’s score on the 
first scale (Content and 
Development) was multiplied by 
two and added to the scores 
achieved on the other two scales 
(Use of Language and 
Conventions). Thus, the total 
possible score for written-response 


. is 20. The reading and writing 


components of the test were given 


_ equal weighting in calculating the 


total test score. To calculate a 
student’s total score, the written- 
response score was multiplied by 
50/20, the reading score was 
multiplied by 50/40, and the two 


awarded on the first scale (State 
and Support an Opinion/Position) 
is multiplied by two and added to 
the score awarded on the second 
scale (Quality of Language and 
Expression). The maximum 
possible score is 15 for written 
response. 


The two major reporting categories 
for the test are knowledge and 
skills. The knowledge score is 
based on multiple-choice items 
only. The skills score is a weighted 
combination of multiple-choice and 
written-component scores. 


Total test scores were calculated 
according to the formula indicated 


scores were added to give a total in the following table. 
based only on the responses of score out of 100. 
Number of Multiple-Choice Multiple Written Total 
Questions Choice Response; = (multiple-choice score) x 
Test Form (70% of Total) Weighting (30% of weighting factor + 2 x 

Knowledge Skills Factor total) written response 

Regular and Form F 24 23 70/47 15 100 

Form E 20 30 70/50 15 100 

Form D 25 25 70/50 15 100 
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Because the knowledge component 
consists entirely of scores on 
multiple-choice items, the 
knowledge scores are reported as 
unweighted raw scores. When 
scores on forms E and D are 
equated to the regular form, the 
equated knowledge scores have the 
same maximum score as the 
regular form; that is, 24. 


The skills component includes both 
multiple-choice and written- 
response questions, and is reported 
only as a weighted combination. 
The maximum score on the skills 
component for the regular form 
and Form F is 64. For Form E, the 
maximum score is 72; for Form D, 
it is 65. When scores on forms E 
and D are equated to the regular 
form, the equated skills scores 
have the same maximum score as 
the regular form; that is, 64. 


Comparing Results to 
Average Scores 

To determine if students are “doing 
well enough,” achievement tests 
are developed in reference to 
curriculum criteria with 
established achievement and 
assessment standards. 
Consequently, most attention is 
given to comparing achievement 
with standards. 


However, in addition to including 
the percentage of students meeting 
standards in school and 
jurisdiction reports, each report 
also provides jurisdiction or school 
average scores for each reporting 
category or subtest. Comparisons 
of each of these scores to the 
provincial average for the same 
reporting category or subtest can 
help determine relative strengths 
and weaknesses of the school or 
jurisdiction compared to results 
achieved provincially. 


Evaluating the importance of and 
reasons for differences between 
jurisdiction or school averages and 
provincial averages requires 
careful consideration of the 
practical significance of 
differences. Also it is important to 
consider local targets for student 
achievement relative to Provincial 
Achievement Standards and 
averages. 


Factors Limiting the 
Interpretation of 
Test Results 


Educators who are interpreting 
results must take into account the 
following limitations: 


1. Paper and pencil tests 
necessarily measure reading 
achievement in the content 
area being tested. Standards 
built into each achievement 
test reflect the reading level 
expectation for the grade level 
tested. Jurisdictions should 
consider the average reading 
level of their grades 3, 6, and 9 
students, as reading levels 
below these grades will have 
an effect on test results. 


2. If more than 10% of eligible 
students in a jurisdiction did 
not write a test, the reported 
statistics for that jurisdiction 
may not accurately represent 
the true level of achievement. 


School Factors Which 
Affect Student 
Achievement 

Research* in education has 
identified key aspects of school 


effectiveness which affect student 
achievement: 


1. Productive School Climate and 
Culture 


there is a shared and 
articulated focus on 
achievement 

there is a shared belief that 
all students can achieve 

e staff is cohesive, 
collaborates, and makes 
decisions by consensus 


. Focus on Student Acquisition 


of Central Learning Skills 

® teachers know what 
students are to learn and 
emphasize mastery of key 
concepts 

students know what is 
expected of them 

learning time is maximized 


. Appropriate Monitoring of 


Student Progress 

e students can show what 
they have learned 

® parents know what their 
child has achieved 


. Outstanding Leadership 


* effective instructional 
leadership is provided 


. Parent Involvement 


¢ high levels of school and 
home cooperation are 
evident 


. Effective Instruction 


grouping and organizational 
arrangements are 
appropriate 

® pacing is appropriate 

* curriculum and learning are 
aligned 

teachers use a variety of 
strategies 

students are actively 
involved 


. High Expectations and 


Requirements for Students 

® students are held 
responsible for learning 

¢ higher order learning is 
emphasized 


*Haertel, Geneva D. et al. (1990) “What Influences Learning? A Content Analysis of Review Literature,” The Journal of Educational 


Research, Vol. 84(1), 30-43 


Levine, Daniel U. & Lezotte, Lawrence W. Unusually Effective Schools (National Center for Effective Schools Research 


& Development, Madison, WI: 1990) 
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Many contextual aspects can be 
considered in interpreting results 
and planning for improved 
learning. These include students’ 
abilities, attitudes, motivations, 
aspirations, academic 
backgrounds, and learning styles. 
They also include students’ family 
circumstances, socioeconomic 
backgrounds, and community 
environments. 


A Systematic Approach 
for Effectively Using Test 
Results 


Educators can use achievement 
test results constructively as one 
means of improving the quality of 
education. It is desirable to involve 
all those affected by achievement 
tests, including teachers, parents, 
and community members, in the 
analysis of test results. A 
systematic use of the test results 
includes the following steps: . 


1. Evaluating the percentage of 
students who wrote the 
achievement tests 
(participation rate) for your 
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school or jurisdiction. If your 
participation rate varies 
greatly from the provincial 
participation rate, you should 
consider the degree to which 
your results reflect 
achievement in your school or 
jurisdiction. 


. Comparing test results to 


province-wide standards. 
Comparisons should include 
results for both major 
components, gender, age, and 
levels information. Provincial 
results are provided so that 
you can see how the whole 
province compares to these 
same standards. 


. Examining the results of 


multiple-choice and numeric- 
response questions. These 
individual question results 
provide an opportunity to 


assess how well your students 


mastered specific curriculum 
standards. 


. Examining the results of 


written-response scoring 
categories. 


5. 


a 


Noting any patterns, 
anomalies, and/or 
interrelationships in the 
results. 


Hypothesizing relationships 
between your observations and 
the factors affecting 
achievement in your school or 
jurisdiction. 


. Developing and implementing 


a plan to improve the quality 
of education for students. 


. Communicating to parents the 


results of achievement tests 
and what the school is 
planning to do to improve 
student achievement. 


Appendix DD 
Reporting to Parents: 


Answers to Frequently. 
Asked Questions 


Vs 


What are the achievement tests? 
The achievement tests are 
provincial government tests 
administered in Alberta schools to 
students in language arts/language 
learning, social studies, science, 
and mathematics. 


What is the purpose of the 
achievement tests? 

The achievement tests help 
Alberta Education to communicate 
provincial expectations and results 
for levels of student performance in 
language arts/language learning, 
social studies, science, and 
mathematics. The tests enable 
Alberta Education to monitor the 
level of achievement of students 
throughout Alberta. The results 
also help local school boards, 
principals, and teachers identify 
the strengths and weaknesses in 
their implementation of these 
programs. 


How many achievement tests will 
my child have to write? 

In 1993, students wrote only one 
test in Grade 3 English Language 
Learning, one in Grade 6 Social 
Studies, and one in Grade 9 
Science. In 1994, the program will 
assess Grade 3 students in 
mathematics, Grade 6 students in 
science, and Grade 9 students in 
language arts. 


How should I prepare my child to 
write an achievement test? 

No preparation beyond normal 
classroom instruction is required to 
write an achievement test. While 


students should be encouraged to 
do their best, a good night’s sleep 
and a relaxed, confident approach 
to testing are the best possible 
preparation. 


How much do these tests count 

for my child? 

The achievement tests do not 
affect students’ final marks. The 
classroom teacher is responsible for 
evaluating students and awarding 
final marks. Achievement test 
results are not released by Alberta 
Education until September, long 
after students’ marks have been 
determined by the classroom 
teacher. 


How do achievement test results 
help classroom teachers? 
Achievement test results provide 
feedback on student achievement 
to school boards, principals, and 
teachers. For example, teachers in 
a school where student performance 
is high in one skill area but low in 
another may wish to examine their 
programs to see if changes are 
needed to achieve a better 
instructional balance. 


What are the limitations of the 
achievement tests? 

Paper and pencil tests cannot 
easily measure such things as 
laboratory skills, small group 
discussions, and creative thinking. 
Thus, some student strengths can 
be evaluated only by the classroom 
teacher. Also, a single test cannot 
reveal as much about a student’s 
development and growth as can 


evaluation by the classroom 
teacher over the course of a full 
school] year. 


What advantage do achievement 
tests have over other standardized 
tests? 

Unlike commercially developed 
tests, achievement tests are based: 
on learning expectations and 
standards communicated through 
Alberta’s programs of study. The 
tests are designed, written, and 
evaluated by experienced 
classroom teachers from across 
the province. Tests developed 
elsewhere may not reflect. 
curriculum or standards 
appropriate for Alberta. 


How do I interpret achievement test 
results? 

The Achievement Testing Program 
Provincial Report includes 
guidelines for interpreting results. 
Readers are cautioned not to 
overgeneralize conclusions based 
on a single administration of the 
test. Results should be compared 
with expectations or with the 
results of previous achievement 
tests in the same subject. Any 
trends that are observed in the 
scores must then be interpreted in 
the context of a variety of factors 
that could affect student 
achievement. 


Comparisons between districts, 
schools, or classrooms are likely to 
prove misleading and are therefore 
discouraged. 
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Can I find out how my child did on 
the achievement test? 

Yes, by contacting the school 
where your child wrote the test. 
Individual results on the 


- achievement tests are made 


available to school principals in the 
fall. Since the tests are designed to 
gather information on groups of 
students, not on individuals, 
individual results must be 
interpreted with caution. Schools 
are expected to share their results 
with parents and the local 
community and discuss follow-up 
plans for improvement. A copy of 
the school report (without the 
individual student scores) should 
be available. 


Where can I get additional 
information about the Achievement 
Testing Program? 

Bulletins describing the content of 
the coming year’s achievement 
tests and the Provincial Report 
describing the results of the 
previous year’s testing are 
distributed to schools each year. 
Requests for copies of these 
publications or questions and 
comments regarding the 
Achievement Testing Program 
should be directed to: 


Mr. Dennis Belyk 

Assistant Director 

Achievement Testing and 
Diagnostic Evaluation Programs 
Student Evaluation Branch 
Alberta Education 

Devonian Building, West Tower 
11160 Jasper Avenue 
Edmonton, Alberta T5K 0L2 
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Appendix E 


Developing 
Achievement Tests 


The Student Evaluation Branch 
develops achievement tests that 
measure student achievement at 
the grades 3, 6, and 9 levels. 
Province-wide testing in language 
arts/language learning, 
mathematics, science, and social 
studies follows a four-year cycle for 
each grade level and subject. Many 
individuals and groups are 
involved in the 

development of each test: 
practising classroom teachers, 
school and central office 
administrators, and representatives 
of post-secondary institutions, the 
Curriculum Branch, the Language 
Services Branch, Regional Offices, 
and the Student Evaluation 
Branch. Student Evaluation 
Branch staff ensure the 
development of valid and reliable 
tests. 


The following is a summary of the 
phases of the test development 
process: 


¢ Planning 
¢ Approving Blueprints 
* Developing Test Questions 


¢ Constructing and Administering 
Field Tests 


e Analyzing and Revising 
¢ Constructing Final Field Tests 
e Approving Final Field Tests 


e Administering Final Field 
Tests 


¢ Constructing the Final Test 


e Preparing and Administering the 
Final Test 


¢ Marking 


¢ Analyzing and Reporting the 
Results 


Under normal circumstances, it 
takes three years to complete all 
phases of the process. 


Planning 


Test developers ensure that the 
design of each achievement test 
reflects the learning expectations 
in the Program of Studies for each 
subject. Planning takes into 
consideration those parts of the 
program that are testable in a 
paper and pencil format, within a 
given time frame. Teachers and 
consultants from across the 
province assist in preparing the 
design of each test. 


Test developers prepare an interim 
test blueprint (an overall plan used 
to guide the development of a test). 
Questions that must be addressed 
at this point are: 


¢ What knowledge, skills, and 
attitudes should students be 
expected to demonstrate? 


¢ What types of questions will 
constitute the test (machine- 

. scored, short answer, or 
extended written response)? 


¢ What weighting will each part 
of the test be given? 


¢ How long and how demanding 
should the test be? 


© How should the results of the 
test be organized for reporting 
purposes? 


In order to ensure that each test 
will produce meaningful and 
reliable results, test developers 
incorporate statistical as well as 
curricular standards in the test 
design. Statistical standards 
address areas such as range of 
question difficulty and the 
requirements for reporting. 


Each dimension of the curriculum 
for which results are reported must 
contain at least six questions if the 
results are to be meaningful. 


Approving Blueprints 


Blueprint approval establishes the 
overall design of the test, the exact 
emphases given to each category 
for which results are reported, and 
the emphases given to the different 
cognitive levels. 


The interim blueprint is reviewed 
by a committee of Alberta 
Education personnel that 
represents the Curriculum Branch 
(or Language Services Branch), 
Regional Office consultants, and 
the Student Evaluation Branch. 
This committee makes 
recommendations to the Director 
of the Student Evaluation Branch. 
The blueprint recommended by 
the Alberta Education committee 
is then reviewed by a Test Review 
Committee, which consists of 
teachers and members nominated 
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by the Conference of Alberta 
School Superintendents and post- 
secondary institutions. This 
committee makes recommendations 
to the Director of the Student 
Evaluation Branch. 


Developing Test 
Questions 


Following blueprint approval, 
committees of practising classroom 
teachers working at the 
appropriate grade level are 
formed, and question development 
meetings are held. These 
committees develop new test 
questions that reflect the learning 
expectations of the Program of 
Studies and curriculum 
specifications. Where necessary, 
question developers are trained 

in the principles of question 
construction. Questions built in 
committee are then screened for 
format, validity, blueprint “fit”, 
and other design considerations. 


Constructing and 
Administering Field 
Tests 


After careful editing and 
formatting of questions developed 
by the teacher committees, field 
tests are constructed. Any required 
artwork is completed during this 
phase of the test development 
process. 


With permission from school and 
jurisdiction personnel, field tests 
are sent to a number of teachers 
throughout Alberta. The students 
involved are representative of the 
student population for which the 
test has been designed. A 
minimum sample of 150 students 
writes each field test. 
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Teachers who administer a field 
test are asked to comment in 
writing on the following: 


* reading level 


¢ how closely the question matches 
the way in which a concept was 
taught 


* level of difficulty of the questions 


* quality of the questions and 
graphics 


e errors of any kind 


The results from the administration 
of this initial round of field tests 
are used to validate content, to 
determine difficulty levels, and to 
ensure that questions are 
expressed clearly. Special field 
tests are also constructed to “try 
out” new assessment strategies 
and techniques that may be useful 
for future assessments. 


Analyzing and Revising 
The results of each field test are 
then analyzed and scrutinized to 
determine whether individual 
questions require revision. 
Teacher comments regarding the 
way that test questions are 
structured and the way that a 
subject is being taught are also 
carefully considered and used to 
guide revision. 


Questions deemed to require 
changes are revised and submitted 
for further field testing. 


Constructing Final 
Field Tests 


Once the initial field test results 
are thoroughly analyzed and 
questions requiring changes are 
revised, final field tests are 
constructed. These field tests 
follow the approved blueprint and 
parallel the actual achievement 
test in format and design. 

Final field tests, like all field 
tests, are submitted for further 
validity checking, editing, and 
proofreading. In grades 6 and 9, 
separate tests in English and in 
French are developed for language 
arts. At this point, all other tests 
for Grade 6 and Grade 9 are 
translated into French. 


Approving Final 
Field Tests 


After the final field tests have been 
constructed, a second meeting of 
the Alberta Education Committee 
that represents the Curriculum 
Branch (or Language Services 
Branch), Regional Office 
consultants, and the Student 
Evaluation Branch is convened. 
This committee reviews the 

final field tests and makes 
recommendations for improvement. 
The Test Review Committee, which 
approved the blueprint in Phase 
Two of the test development 
process, meets a second time to 
review and recommend for 
approval the final field tests and 
the instructions for administering 
the tests. Ifa test includes short- 
answer or extended-writing 
questions, the Test Review 
Committee discusses standards 

of achievement and marking 
standards appropriate for the test. 
Again, this committee makes 
recommendations to the Director 
of the Student Evaluation Branch. 


Administering Final 
Field Tests 


The final field tests are 
administered and the results are 
used as a final screen in selecting 
questions for placement on the 
provincial achievement test. A 
minimum sample of 250 students 
writes each final field test. The 
sample is selected to include: 


* only students who have received 
instruction in the course 


* students representing a normal 
distribution of ability levels 


students from rural and urban 
schools 


¢ students from large and small 
schools 


Constructing the 
Final Test 


The construction of the final test 
form is based upon information 
collected from the final field test 
administration. The Test Review 
Committee is reconvened to review 
the final test form and to assist in 
setting assessment and 
achievement standards. 


The test is submitted for final 
validity checking, editing, and 
proofreading. Grade 6 and Grade 9 
achievement tests, in subjects 
other than language arts, are 
translated into French. 


For each test, an information 
bulletin is prepared that outlines 
the design and nature of the 
upcoming tests. These bulletins are 
distributed to each school at the 
beginning of the school year 

to facilitate program and 
instructional planning by teachers 
and administrators. 


Preparing and 
Administering the 
Final Test 


The completed achievement test is 
commercially printed and prepared 
for distribution. Sufficient copies of 
the test are mailed to each school. 
Quantities are based:on the 
number of students enrolled in the 
subject as reported to the Student 
Evaluation Branch. The test is 
administered to students by their 
classroom teachers. 


Marking 


All written-response sections of 
the tests are marked by classroom 
teachers. These teachers, who 

are recommended by their 
superintendents, are currently 
teaching the course being 
evaluated, have taught the course 
for a minimum of two years, and 
hold a valid Alberta Permanent 
Professional Certificate. 


Student Evaluation Branch staff 
train and supervise the teachers 
during the marking sessions. 

All multiple-choice and numerical 
responses are machine scored. 


Analyzing and Reporting 
the Results 


A results report is prepared and 
distributed to superintendents, 
school principals, Alberta 
Education officials, and other 
Departments of Education. This 
report is also made available to the 
general public. In addition to the 
Achievement Testing Program 
Provincial Report, each school and 
jurisdiction receives a statistical 
summary for its student 
population. 


For further information, please 
refer to the Achievement Test 
Bulletins or call the Assistant 
Director, Achievement Testing and 
Diagnostic Evaluation Programs, 
at 427-0010. 
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1993 Achievement Testing 


Program Provincial Report 


The Student Evaluation Branch 
strives to produce documents 

that are useful to educators and 
their communities. The purpose 


and format of the next report are 
reviewed. Please take a moment to 
respond to the questions and send 
to: Assistant Director, Analytic 


of the following questionnaireis_ . Services 
to collect your opinions about the Student Evaluation Branch 
Achievement Testing Program Alberta Education 
‘ ‘s Provincial Report. All opinions 11160 Jasper Avenue 
Que s tionnair e are considered when the content Edmonton, Alberta T5K 0L2 
Content of the Report 
1. Please judge the usefulness of the information included in the various sections of the report by checking 
the appropriate boxes below. 
Very Adequately Partially Not 
Useful Useful Useful Useful 
Section 1: Summary of Achievement Test Results 
Section 2: Grade Level 3 Language Learning L [s-s=| est] 
Section 3: Grade 6 Social Studies Ee 
Section 4: Grade 9 Science Ut] La] 
Section 5: Achievement by Gender = — — — 
Section 6: Achievement by Age Ea) [| L__} L__| 
Section 7: Achievement by Grade Level — L__| L___d L__| 
Section 8: Achievement Over Time [| Cas L__| L__4i 
Section 9: Age 9 Achievement L [| L__| L____| 
Appendix A: Standards 
Appendix B: Public Review of Standards and Results | 
Appendix C: Guidelines for Interpreting Results | 
Appendix D: Reporting to Parents—Answers to 
Frequently Asked Questions 
Appendix E: Developing Achievement Tests 
Format of the Report 
1. Please judge the usefulness of the report’s format by checking the appropriate boxes below. 


Organization into Separate 


Sections 


Three-Column Presentation 


of Text 


Presentation of Figures 


Presentation of Tables 
Blending of Information 


in Text, Figures, and Tables 


Very 
Useful 


Adequately Partially Not 
Useful 


Useful Useful 


Please write on the back of this page if you wish to comment further on the report. 
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